Affordable solution for Jenkins CI and build slaves in the cloud?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Affordable solution for Jenkins CI and build slaves in the cloud?

Eddie Sutton
I no longer want to deal with backup issues of our CI system.

I have an ESXi server running on a Mac Mini hosting a Jenkins server and multiple build machines on a Synology NAS datastore targeting Android, iOS, Linux, macOS, and Linux. 

Any success stories or recommendations?

Thanks in advance,

-Ed

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CAKas9SStLd2_kxOWMt3uptwGQ0EzYsf7VZf4gCmgMxF4guf4HQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Affordable solution for Jenkins CI and build slaves in the cloud?

Victor Martinez

If you mean to backup your Jenkins server, then read below:

   In my case I've been using tools such as:
  • Puppet: https://github.com/jenkinsci/puppet-jenkins
  • Chef: https://github.com/chef-cookbooks/jenkins
  • Ansible: https://github.com/geerlingguy/ansible-role-jenkins
        NOTE: You can use docker (https://hub.docker.com/r/jenkins/jenkins/) although the complexity will be with a live and continuous up to date instance, in my opinion. 

   The second part is about using a subset of the Configuration as Code tools such as:
  • Pipelines: https://jenkins.io/doc/book/pipeline/
  • JobDSL: https://github.com/jenkinsci/job-dsl-plugin/wiki
  • Jenkins Job Builder: https://docs.openstack.org/infra/jenkins-job-builder/
    Finally, you can automate the last bits and pieces using Groovy and the https://wiki.jenkins.io/display/JENKINS/Post-initialization+script

The above suggestion is based on the Configuration as Code paradigm and being able to specify the current jenkins instance state either programmatically or with some configuration in code. 

My two cents

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/eca4514c-6eef-4c44-b5e3-edea64aede46%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Affordable solution for Jenkins CI and build slaves in the cloud?

Artur Szostak
> If you mean to backup your Jenkins server, then read below:
>
>    In my case I've been using tools such as:
>
>   *   Puppet: https://github.com/jenkinsci/puppet-jenkins
>   *   Chef: https://github.com/chef-cookbooks/jenkins
>   *   Ansible: https://github.com/geerlingguy/ansible-role-jenkins
>

How is this related to backup? This just deals with installation and configuration management.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/db5a45b637fe4af995082dd0e750bf2d%40partner.eso.org.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Affordable solution for Jenkins CI and build slaves in the cloud?

Victor Martinez
My answer was suggesting how to move from traditional backup processes since there are already tools/frameworks, as you just mentioned, which help to configured everything fully as code. I guess, that's why it's related to backup :)


--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/0e1cc9e0-4bba-4c2d-ab30-025833409363%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Affordable solution for Jenkins CI and build slaves in the cloud?

Artur Szostak
OK, but that does not help with the build logs etc. i.e. If I loose my server, I not only want to recreate the instance, but I want to restore the build logs. This requirement may not be for everyone. But for people who consider the build history valuable it would be important to back that up also.

Unfortunately I do not think that Jenkins has an enterprise grade backup solution available out of the box. You have to put it together by yourself. And then there is no way to do it 100% correctly without a full shutdown of the Jenkins master at some point. Which is unfortunate, since that makes it impossible to have a high availability Jenkins that is also being backed up regularly and automatically at the same time. One can get close with clever use of a copy-on-write snapshotting file systems like Btrfs, but you will not have 100% guarantee that the snapshot is consistent since Jenkins has no quiescing facility.
________________________________________
From: Victor Martinez <[hidden email]>
Sent: 17 August 2017 13:52:08
To: Jenkins Users
Cc: Artur Szostak
Subject: Re: Affordable solution for Jenkins CI and build slaves in the cloud?

My answer was suggesting how to move from traditional backup processes since there are already tools/frameworks, as you just mentioned, which help to configured everything fully as code. I guess, that's why it's related to backup :)


--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/2022a80edcf74ceaacc3c68a3e4eaf61%40partner.eso.org.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Affordable solution for Jenkins CI and build slaves in the cloud?

corneil.duplessis@gmail.com
How about using JFrog Artifactory in the cloud and configure Jenkins to push build logs to Artifactory https://www.jfrog.com/confluence/display/RTF/Build+Integration#BuildIntegration-SupportedPlugins


On 17 August 2017 at 15:52, Artur Szostak <[hidden email]> wrote:
OK, but that does not help with the build logs etc. i.e. If I loose my server, I not only want to recreate the instance, but I want to restore the build logs. This requirement may not be for everyone. But for people who consider the build history valuable it would be important to back that up also.

Unfortunately I do not think that Jenkins has an enterprise grade backup solution available out of the box. You have to put it together by yourself. And then there is no way to do it 100% correctly without a full shutdown of the Jenkins master at some point. Which is unfortunate, since that makes it impossible to have a high availability Jenkins that is also being backed up regularly and automatically at the same time. One can get close with clever use of a copy-on-write snapshotting file systems like Btrfs, but you will not have 100% guarantee that the snapshot is consistent since Jenkins has no quiescing facility.
________________________________________
From: Victor Martinez <[hidden email]>
Sent: 17 August 2017 13:52:08
To: Jenkins Users
Cc: Artur Szostak
Subject: Re: Affordable solution for Jenkins CI and build slaves in the cloud?

My answer was suggesting how to move from traditional backup processes since there are already tools/frameworks, as you just mentioned, which help to configured everything fully as code. I guess, that's why it's related to backup :)


--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/2022a80edcf74ceaacc3c68a3e4eaf61%40partner.eso.org.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CACPng9bi46KpfATf1bw%2B4zv-NAMkVnjvQti3dystLdxsk%3DNfhw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Affordable solution for Jenkins CI and build slaves in the cloud?

Victor Martinez
Yep, I agree you might need to either use the current backup plugins:
- https://wiki.jenkins.io/display/JENKINS/thinBackup
- https://wiki.jenkins.io/display/JENKINS/PeriodicBackup+Plugin

Or create your own backup process.

Although, I just found that Cloudbees provides an enterprise plugin which already sync with AS3 among others:
- https://go.cloudbees.com/docs/cloudbees-documentation/cje-user-guide/index.html#backup

Cheers

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/c40ab023-d74e-406d-b39f-a355a03760a6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Loading...