Build Docker Image from a Dockerized Jenkins

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

Build Docker Image from a Dockerized Jenkins

Jeeva Chelladhurai
Hello All,

I would like to build Docker images from a Dockerized Jenkins. There are multiple solution out there but what is the recommended solution...

I have been using ssh node as a work around, but I was told to use https://wiki.jenkins.io/display/JENKINS/Docker+Slaves+Plugin . In order for this plugin to work, I have to install docker inside the Jenkins (my docker host is RHEL) and also mount the /var/run/docker.sock or enable TLS. Isn't installing docker blow up the size of the Jenkins image? 

Any way to install just the docker client in side the Jenkins image? 

Thanks,
Jeeva

--
Jeeva K S Chelladhurai

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CAK5CbZV3Sc-fgYS_34PgEwRzsF20%3Dp6mbaagbR2Ehtz01op2%3Dw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Build Docker Image from a Dockerized Jenkins

Jacob Larsen-2

I wouldn't use Docker Slaves for this. Just install the docker client in your Jenkins image and bind mount /var/run/docker.sock. Then a simple sh step with docker build + docker push should do the trick.

If you don't want to use your jenkins master for this, you could make a slave container with these tools and bind mount.

The Docker Slaves plugin is more about running your builds in containers than it is about building images.

/Jacob


On 2017-12-03 18:42, Jeeva Chelladhurai wrote:
Hello All,

I would like to build Docker images from a Dockerized Jenkins. There are multiple solution out there but what is the recommended solution...

I have been using ssh node as a work around, but I was told to use https://wiki.jenkins.io/display/JENKINS/Docker+Slaves+Plugin . In order for this plugin to work, I have to install docker inside the Jenkins (my docker host is RHEL) and also mount the /var/run/docker.sock or enable TLS. Isn't installing docker blow up the size of the Jenkins image? 

Any way to install just the docker client in side the Jenkins image? 

Thanks,
Jeeva

--
Jeeva K S Chelladhurai

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CAK5CbZV3Sc-fgYS_34PgEwRzsF20%3Dp6mbaagbR2Ehtz01op2%3Dw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/2162124b-1539-acc7-e92c-56ac5674aa7c%40larsen.net.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Build Docker Image from a Dockerized Jenkins

nicolas de loof-2
In reply to this post by Jeeva Chelladhurai
There's no "recommended" solution, each plugin comes with it's own vision, benefits and drawbacks.

docker-plugin can be used to run dockerized agents, it can run from a dockerized jenkins master with /var/run/docker.sock without docker CLI installed, as it uses a java client library for docker API.
docker-slaves-plugin do rely on running docker CLI executed from master, I planned to use a java API at some point but still need to work on this. In the meantime I don't think there's a simple way to just install docker CLI. Installing full docker will make your image a bit bigger with additional files on filesystem but there won't be any extra process running, so not a major issue (considering jenkins docker image is already a big one)

2017-12-03 18:42 GMT+01:00 Jeeva Chelladhurai <[hidden email]>:
Hello All,

I would like to build Docker images from a Dockerized Jenkins. There are multiple solution out there but what is the recommended solution...

I have been using ssh node as a work around, but I was told to use https://wiki.jenkins.io/display/JENKINS/Docker+Slaves+Plugin . In order for this plugin to work, I have to install docker inside the Jenkins (my docker host is RHEL) and also mount the /var/run/docker.sock or enable TLS. Isn't installing docker blow up the size of the Jenkins image? 

Any way to install just the docker client in side the Jenkins image? 

Thanks,
Jeeva

--
Jeeva K S Chelladhurai

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CAK5CbZV3Sc-fgYS_34PgEwRzsF20%3Dp6mbaagbR2Ehtz01op2%3Dw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CANMVJzk7GJDmKUKQNvt4PzLpXHtTp%2B7BnikYfeysX7_kdpxcUA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Build Docker Image from a Dockerized Jenkins

Jeeva Chelladhurai
Hi Nicolas,

Thanks for your input. Using Java API is an excellent idea. There is hackathon planned for Jenkins this week https://www.meetup.com/jenkinsBLR/events/240212894/
Perhaps, this requirement could be prioritized at the hackathon if you could give a helping hand..

Thanks,
Jeeva

On Mon, Dec 4, 2017 at 12:48 AM, nicolas de loof <[hidden email]> wrote:
There's no "recommended" solution, each plugin comes with it's own vision, benefits and drawbacks.

docker-plugin can be used to run dockerized agents, it can run from a dockerized jenkins master with /var/run/docker.sock without docker CLI installed, as it uses a java client library for docker API.
docker-slaves-plugin do rely on running docker CLI executed from master, I planned to use a java API at some point but still need to work on this. In the meantime I don't think there's a simple way to just install docker CLI. Installing full docker will make your image a bit bigger with additional files on filesystem but there won't be any extra process running, so not a major issue (considering jenkins docker image is already a big one)

2017-12-03 18:42 GMT+01:00 Jeeva Chelladhurai <[hidden email]>:
Hello All,

I would like to build Docker images from a Dockerized Jenkins. There are multiple solution out there but what is the recommended solution...

I have been using ssh node as a work around, but I was told to use https://wiki.jenkins.io/display/JENKINS/Docker+Slaves+Plugin . In order for this plugin to work, I have to install docker inside the Jenkins (my docker host is RHEL) and also mount the /var/run/docker.sock or enable TLS. Isn't installing docker blow up the size of the Jenkins image? 

Any way to install just the docker client in side the Jenkins image? 

Thanks,
Jeeva

--
Jeeva K S Chelladhurai

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CAK5CbZV3Sc-fgYS_34PgEwRzsF20%3Dp6mbaagbR2Ehtz01op2%3Dw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CANMVJzk7GJDmKUKQNvt4PzLpXHtTp%2B7BnikYfeysX7_kdpxcUA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.



--
Jeeva K S Chelladhurai

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CAK5CbZUpZCQDctutTX9BwouEkT7Jnf%2BOLzXejpqFHKkK_ON%3DLQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Build Docker Image from a Dockerized Jenkins

Ramanathan Muthaiah
In reply to this post by Jeeva Chelladhurai
Hi Jeeva,
 
I would like to build Docker images from a Dockerized Jenkins. There are multiple solution out there but what is the recommended solution...

I have been using ssh node as a work around, but I was told to use <a href="https://wiki.jenkins.io/display/JENKINS/Docker+Slaves+Plugin" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fwiki.jenkins.io%2Fdisplay%2FJENKINS%2FDocker%2BSlaves%2BPlugin\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNH0gOhowfAVYWyJyQSIo0ByOId54g&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fwiki.jenkins.io%2Fdisplay%2FJENKINS%2FDocker%2BSlaves%2BPlugin\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNH0gOhowfAVYWyJyQSIo0ByOId54g&#39;;return true;">https://wiki.jenkins.io/display/JENKINS/Docker+Slaves+Plugin . In order for this plugin to work, I have to install docker inside the Jenkins (my docker host is RHEL) and also mount the /var/run/docker.sock or enable TLS. Isn't installing docker blow up the size of the Jenkins image? 

Any way to install just the docker client in side the Jenkins image? 

I have not tried this, so, cannot vouch for this. Please be cautious about the approach presented in this blog.

http://niels.nu/blog/2017/continuous-blog-delivery-p2.html : you might be interested in Docker-in-Docker.

/Ram 

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/e973fe60-490e-476c-9efb-de2aa5a2e09c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Build Docker Image from a Dockerized Jenkins

Jeeva Chelladhurai


I have not tried this, so, cannot vouch for this. Please be cautious about the approach presented in this blog.

http://niels.nu/blog/2017/continuous-blog-delivery-p2.html : you might be interested in Docker-in-Docker.


Thanks for the suggestion.. Docker-in-Docker is been discouraged here is good read on it https://jpetazzo.github.io/2015/09/03/do-not-use-docker-in-docker-for-ci/

- Jeeva

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CAK5CbZXcvHWevqGmjPTXkAhU5MRNuHk6LOg6iEYrMwdNyb-abA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Build Docker Image from a Dockerized Jenkins

Jeeva Chelladhurai

On Mon, Dec 4, 2017 at 12:04 PM, Jeeva Chelladhurai <[hidden email]> wrote:


I have not tried this, so, cannot vouch for this. Please be cautious about the approach presented in this blog.

http://niels.nu/blog/2017/continuous-blog-delivery-p2.html : you might be interested in Docker-in-Docker.


Thanks for the suggestion.. Docker-in-Docker is been discouraged here is good read on it https://jpetazzo.github.io/2015/09/03/do-not-use-docker-in-docker-for-ci/

- Jeeva



--
Jeeva K S Chelladhurai

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CAK5CbZVSDwjOAUe_uB_u89GRufSrbk2MyeUzSHz9w88K6af0QA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Build Docker Image from a Dockerized Jenkins

nicolas de loof-2
(another) plugin of mines.
Same design as docker-pipeline's "docker.inside"

Can you please describe your use-case so we give better advise ?

if you just want to "build Docker images" as your initial message told, then just run "docker build" within your job, what else ?

2017-12-04 13:15 GMT+01:00 Jeeva Chelladhurai <[hidden email]>:

On Mon, Dec 4, 2017 at 12:04 PM, Jeeva Chelladhurai <[hidden email]> wrote:


I have not tried this, so, cannot vouch for this. Please be cautious about the approach presented in this blog.

http://niels.nu/blog/2017/continuous-blog-delivery-p2.html : you might be interested in Docker-in-Docker.


Thanks for the suggestion.. Docker-in-Docker is been discouraged here is good read on it https://jpetazzo.github.io/2015/09/03/do-not-use-docker-in-docker-for-ci/

- Jeeva



--
Jeeva K S Chelladhurai

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CAK5CbZVSDwjOAUe_uB_u89GRufSrbk2MyeUzSHz9w88K6af0QA%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CANMVJznKg%3DXKpeaT62PVkmEvLh4kruCyc1HV%2BQGFNtnjLhX%3DLQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Build Docker Image from a Dockerized Jenkins

Jeeva Chelladhurai
I would like to run my builds inside docker, so that I can manage different run times like node, python, go thru docker images.

- Jeeva

On Mon, Dec 4, 2017 at 5:51 PM, nicolas de loof <[hidden email]> wrote:
(another) plugin of mines.
Same design as docker-pipeline's "docker.inside"

Can you please describe your use-case so we give better advise ?

if you just want to "build Docker images" as your initial message told, then just run "docker build" within your job, what else ?

2017-12-04 13:15 GMT+01:00 Jeeva Chelladhurai <[hidden email]>:

On Mon, Dec 4, 2017 at 12:04 PM, Jeeva Chelladhurai <[hidden email]> wrote:


I have not tried this, so, cannot vouch for this. Please be cautious about the approach presented in this blog.

http://niels.nu/blog/2017/continuous-blog-delivery-p2.html : you might be interested in Docker-in-Docker.


Thanks for the suggestion.. Docker-in-Docker is been discouraged here is good read on it https://jpetazzo.github.io/2015/09/03/do-not-use-docker-in-docker-for-ci/

- Jeeva



--
Jeeva K S Chelladhurai

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CAK5CbZVSDwjOAUe_uB_u89GRufSrbk2MyeUzSHz9w88K6af0QA%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CANMVJznKg%3DXKpeaT62PVkmEvLh4kruCyc1HV%2BQGFNtnjLhX%3DLQ%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.



--
Jeeva K S Chelladhurai

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CAK5CbZVoig-s%3DvfVi5SEcrLVjBwZ5-0danHP-X54_CogRT9n0Q%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Build Docker Image from a Dockerized Jenkins

Jacob Larsen-2

Here is a list of the options I know, but one important aspect to this is if you use pipeline or not in your builds as that affects your choices:

Docker Custom Build Environment Plugin. This will use an existing executor to launch a container with the build. The images used need not be aware of Jenkins or dependencies. When I looked at it, it seemed abandoned, but it looks like someone is trying to pick it up again. It will probably take some time before it is up to date with new docker and Jenkins features. E.g. last I checked, pipeline was not supported. Also, it only works from physical Jenkins nodes.

Docker Slaves Plugin. This will provision slave containers on the fly from a single Docker host. Images need not be aware of Jenkins or dependencies. Limit on containers launched did not work for pipeline jobs last I checked. Also, this only works on a single Docker host.

Docker Pipeline Plugin https://plugins.jenkins.io/docker-workflow. I just learned about this one from the previous message, but looking at the adoption rate, this must be the mainstream option. Odds are this would be your best bet to start, provided you are using pipeline.

Using a build wrapper like SCUBA: https://github.com/JonathonReinhart/scuba in a shell step. This has the benefit of enabling manual/development builds to use the same steps as the automated Jenkins builds. For that particular wrapper, it does not support running from inside a container and creating sibling containers. I am considering a PR for that, but I think there are other wrappers that might support something like this.

There are plugins that dynamically provision slaves from images and run builds on those. The common drawback for these would be that the images themselves must support running a jenkins Slave, e.g. have a Java runtime of a certain version installed.

/Jacob


On 2017-12-04 17:37, Jeeva Chelladhurai wrote:
I would like to run my builds inside docker, so that I can manage different run times like node, python, go thru docker images.

- Jeeva

On Mon, Dec 4, 2017 at 5:51 PM, nicolas de loof <[hidden email]> wrote:
(another) plugin of mines.
Same design as docker-pipeline's "docker.inside"

Can you please describe your use-case so we give better advise ?

if you just want to "build Docker images" as your initial message told, then just run "docker build" within your job, what else ?

2017-12-04 13:15 GMT+01:00 Jeeva Chelladhurai <[hidden email]>:

On Mon, Dec 4, 2017 at 12:04 PM, Jeeva Chelladhurai <[hidden email]> wrote:


I have not tried this, so, cannot vouch for this. Please be cautious about the approach presented in this blog.

http://niels.nu/blog/2017/continuous-blog-delivery-p2.html : you might be interested in Docker-in-Docker.


Thanks for the suggestion.. Docker-in-Docker is been discouraged here is good read on it https://jpetazzo.github.io/2015/09/03/do-not-use-docker-in-docker-for-ci/

- Jeeva



--
Jeeva K S Chelladhurai

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CAK5CbZVSDwjOAUe_uB_u89GRufSrbk2MyeUzSHz9w88K6af0QA%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CANMVJznKg%3DXKpeaT62PVkmEvLh4kruCyc1HV%2BQGFNtnjLhX%3DLQ%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.



--
Jeeva K S Chelladhurai

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CAK5CbZVoig-s%3DvfVi5SEcrLVjBwZ5-0danHP-X54_CogRT9n0Q%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/189e04a2-be8d-1052-bb05-cebd482ffc72%40larsen.net.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Build Docker Image from a Dockerized Jenkins

nicolas de loof-2
In reply to this post by Jeeva Chelladhurai
ok, so for this purpose you can rely on docker-plugin to create agents on-demand from configured docker images, or docker-slaves-plugin if you want this configured directly within your jobs, and docker-pipeline "inside" if you prefer Jenkinsfiles to setup CI/CD in jenkins.

2017-12-04 17:37 GMT+01:00 Jeeva Chelladhurai <[hidden email]>:
I would like to run my builds inside docker, so that I can manage different run times like node, python, go thru docker images.

- Jeeva

On Mon, Dec 4, 2017 at 5:51 PM, nicolas de loof <[hidden email]> wrote:
(another) plugin of mines.
Same design as docker-pipeline's "docker.inside"

Can you please describe your use-case so we give better advise ?

if you just want to "build Docker images" as your initial message told, then just run "docker build" within your job, what else ?

2017-12-04 13:15 GMT+01:00 Jeeva Chelladhurai <[hidden email]>:

On Mon, Dec 4, 2017 at 12:04 PM, Jeeva Chelladhurai <[hidden email]> wrote:


I have not tried this, so, cannot vouch for this. Please be cautious about the approach presented in this blog.

http://niels.nu/blog/2017/continuous-blog-delivery-p2.html : you might be interested in Docker-in-Docker.


Thanks for the suggestion.. Docker-in-Docker is been discouraged here is good read on it https://jpetazzo.github.io/2015/09/03/do-not-use-docker-in-docker-for-ci/

- Jeeva



--
Jeeva K S Chelladhurai

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CAK5CbZVSDwjOAUe_uB_u89GRufSrbk2MyeUzSHz9w88K6af0QA%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CANMVJznKg%3DXKpeaT62PVkmEvLh4kruCyc1HV%2BQGFNtnjLhX%3DLQ%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.



--
Jeeva K S Chelladhurai

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CAK5CbZVoig-s%3DvfVi5SEcrLVjBwZ5-0danHP-X54_CogRT9n0Q%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CANMVJznagKiHR4qPv6%2BCBJ-N%3DJ6Q%3Dj6ZDspYtWLGEnNDcP-bSQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Build Docker Image from a Dockerized Jenkins

nicolas de loof-2
In reply to this post by Jacob Larsen-2


2017-12-04 21:38 GMT+01:00 Jacob Larsen <[hidden email]>:

Here is a list of the options I know, but one important aspect to this is if you use pipeline or not in your builds as that affects your choices:

Docker Custom Build Environment Plugin. This will use an existing executor to launch a container with the build. The images used need not be aware of Jenkins or dependencies. When I looked at it, it seemed abandoned, but it looks like someone is trying to pick it up again. It will probably take some time before it is up to date with new docker and Jenkins features. E.g. last I checked, pipeline was not supported. Also, it only works from physical Jenkins nodes.

pipeline is not supported by this plugin as the exact equivalent (even code is a re-implementation) is docker-pipeline's "inside".
 

Docker Slaves Plugin. This will provision slave containers on the fly from a single Docker host. Images need not be aware of Jenkins or dependencies. Limit on containers launched did not work for pipeline jobs last I checked. Also, this only works on a single Docker host.

docker-slaves-plugin do NOT require image to be aware of Jenkins or dependencies, you can use arbitrary docker image, even without a JDK. This was one initial goal of this plugin.

Docker Pipeline Plugin https://plugins.jenkins.io/docker-workflow. I just learned about this one from the previous message, but looking at the adoption rate, this must be the mainstream option. Odds are this would be your best bet to start, provided you are using pipeline.

Using a build wrapper like SCUBA: https://github.com/JonathonReinhart/scuba in a shell step. This has the benefit of enabling manual/development builds to use the same steps as the automated Jenkins builds. For that particular wrapper, it does not support running from inside a container and creating sibling containers. I am considering a PR for that, but I think there are other wrappers that might support something like this.

There are plugins that dynamically provision slaves from images and run builds on those. The common drawback for these would be that the images themselves must support running a jenkins Slave, e.g. have a Java runtime of a certain version installed.

/Jacob


On 2017-12-04 17:37, Jeeva Chelladhurai wrote:
I would like to run my builds inside docker, so that I can manage different run times like node, python, go thru docker images.

- Jeeva

On Mon, Dec 4, 2017 at 5:51 PM, nicolas de loof <[hidden email]> wrote:
(another) plugin of mines.
Same design as docker-pipeline's "docker.inside"

Can you please describe your use-case so we give better advise ?

if you just want to "build Docker images" as your initial message told, then just run "docker build" within your job, what else ?

2017-12-04 13:15 GMT+01:00 Jeeva Chelladhurai <[hidden email]>:

On Mon, Dec 4, 2017 at 12:04 PM, Jeeva Chelladhurai <[hidden email]> wrote:


I have not tried this, so, cannot vouch for this. Please be cautious about the approach presented in this blog.

http://niels.nu/blog/2017/continuous-blog-delivery-p2.html : you might be interested in Docker-in-Docker.


Thanks for the suggestion.. Docker-in-Docker is been discouraged here is good read on it https://jpetazzo.github.io/2015/09/03/do-not-use-docker-in-docker-for-ci/

- Jeeva



--
Jeeva K S Chelladhurai

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CAK5CbZVSDwjOAUe_uB_u89GRufSrbk2MyeUzSHz9w88K6af0QA%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CANMVJznKg%3DXKpeaT62PVkmEvLh4kruCyc1HV%2BQGFNtnjLhX%3DLQ%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.



--
Jeeva K S Chelladhurai

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CAK5CbZVoig-s%3DvfVi5SEcrLVjBwZ5-0danHP-X54_CogRT9n0Q%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/189e04a2-be8d-1052-bb05-cebd482ffc72%40larsen.net.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CANMVJzn7U2NvWGsYrWggNK_-Kc_zM9hE6AYh56BT-c3SHb5Wzw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Build Docker Image from a Dockerized Jenkins

Jacob Larsen-2



On 2017-12-05 09:03, nicolas de loof wrote:


2017-12-04 21:38 GMT+01:00 Jacob Larsen <[hidden email]>:

Here is a list of the options I know, but one important aspect to this is if you use pipeline or not in your builds as that affects your choices:

Docker Custom Build Environment Plugin. This will use an existing executor to launch a container with the build. The images used need not be aware of Jenkins or dependencies. When I looked at it, it seemed abandoned, but it looks like someone is trying to pick it up again. It will probably take some time before it is up to date with new docker and Jenkins features. E.g. last I checked, pipeline was not supported. Also, it only works from physical Jenkins nodes.

pipeline is not supported by this plugin as the exact equivalent (even code is a re-implementation) is docker-pipeline's "inside".

Yes I just discovered this plugin myself. Not sure how I missed it.
 

Docker Slaves Plugin. This will provision slave containers on the fly from a single Docker host. Images need not be aware of Jenkins or dependencies. Limit on containers launched did not work for pipeline jobs last I checked. Also, this only works on a single Docker host.

docker-slaves-plugin do NOT require image to be aware of Jenkins or dependencies, you can use arbitrary docker image, even without a JDK. This was one initial goal of this plugin.

I hope that was what I wrote too. An important feature IMO.

For myself I am moving towards a setup where the docker run is handled by the build logic in the workspace and not by the automation tool/jenkins plugins. I hope this can let development builds get closer to the automated ones.
This involves a one-shot docker run wrapper of some sorts instead of Jenkins plugins. Still investigating though.


Docker Pipeline Plugin https://plugins.jenkins.io/docker-workflow. I just learned about this one from the previous message, but looking at the adoption rate, this must be the mainstream option. Odds are this would be your best bet to start, provided you are using pipeline.

Using a build wrapper like SCUBA: https://github.com/JonathonReinhart/scuba in a shell step. This has the benefit of enabling manual/development builds to use the same steps as the automated Jenkins builds. For that particular wrapper, it does not support running from inside a container and creating sibling containers. I am considering a PR for that, but I think there are other wrappers that might support something like this.

There are plugins that dynamically provision slaves from images and run builds on those. The common drawback for these would be that the images themselves must support running a jenkins Slave, e.g. have a Java runtime of a certain version installed.

/Jacob


On 2017-12-04 17:37, Jeeva Chelladhurai wrote:
I would like to run my builds inside docker, so that I can manage different run times like node, python, go thru docker images.

- Jeeva

On Mon, Dec 4, 2017 at 5:51 PM, nicolas de loof <[hidden email]> wrote:
(another) plugin of mines.
Same design as docker-pipeline's "docker.inside"

Can you please describe your use-case so we give better advise ?

if you just want to "build Docker images" as your initial message told, then just run "docker build" within your job, what else ?

2017-12-04 13:15 GMT+01:00 Jeeva Chelladhurai <[hidden email]>:

On Mon, Dec 4, 2017 at 12:04 PM, Jeeva Chelladhurai <[hidden email]> wrote:


I have not tried this, so, cannot vouch for this. Please be cautious about the approach presented in this blog.

http://niels.nu/blog/2017/continuous-blog-delivery-p2.html : you might be interested in Docker-in-Docker.


Thanks for the suggestion.. Docker-in-Docker is been discouraged here is good read on it https://jpetazzo.github.io/2015/09/03/do-not-use-docker-in-docker-for-ci/

- Jeeva



--
Jeeva K S Chelladhurai

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CAK5CbZVSDwjOAUe_uB_u89GRufSrbk2MyeUzSHz9w88K6af0QA%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CANMVJznKg%3DXKpeaT62PVkmEvLh4kruCyc1HV%2BQGFNtnjLhX%3DLQ%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.



--
Jeeva K S Chelladhurai

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CAK5CbZVoig-s%3DvfVi5SEcrLVjBwZ5-0danHP-X54_CogRT9n0Q%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/189e04a2-be8d-1052-bb05-cebd482ffc72%40larsen.net.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CANMVJzn7U2NvWGsYrWggNK_-Kc_zM9hE6AYh56BT-c3SHb5Wzw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/b78121d2-1ae9-87b1-a74e-e22f3e361164%40larsen.net.
For more options, visit https://groups.google.com/d/optout.