Jenkins Backup and Restore

classic Classic list List threaded Threaded
18 messages Options
Reply | Threaded
Open this post in threaded view
|

Jenkins Backup and Restore

gotviseryon
We have a Jenkins instance running and we are trying to setup a backup and restore mechanism in case of any mishaps.  We are looking for a full backup of the Jenkins and so decided to go with PeriodicBackup plugin.  With this plugin, the backup works fine.  But, we are trying to restore it in a completely new Jenkins setup and not able to restore.  When we pick the backup file that we want to restore from, it says restoring, but nothing happens.  Even after waiting out for over 30 mins, nothing is restored.  As a work around, I stopped the Jenkins service, renamed the 'JENKINS_HOME' (/var/lib/jenkins) folder to 'jenkins_backup' and created another folder called 'jenkins' in the same location.  In this folder, I copied all the files from the backup and then started Jenkins (NOTE: I didn't change the .owner file.  I'm using the .owner file of the new Jenkins only).  Jenkins came up and I was able to see all the configurations from the old one.  Everything was fine.  I let it like that overnight and in the morning when I came and tried to access it, Jenkins was way too slow and I was not able to save any new configuration.  When I try I'm getting 
java.io.ioexception: too many open files jenkins
error.  I restarted the jenkins and it's taking too long for the UI to come up.  Am I doing anything wrong here?  Any suggestions for my requirement?

P.S. - I tried SCMSync plugin and ThinBackup.  But, problem with these are that they don't do full backup.  But, with ThinBackup both backup and restore works fine.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Jenkins Backup and Restore

Rob Pickerill
Hi

Have you taken a look through: https://wiki.jenkins.io/display/JENKINS/I%27m+getting+too+many+open+files+error and https://www.cyberciti.biz/faq/linux-increase-the-maximum-number-of-open-files/ - these give guidance on handling and increasing the open file limit so you can restore your backup.

Thanks

On Thu, 1 Nov 2018 at 20:36, <[hidden email]> wrote:
We have a Jenkins instance running and we are trying to setup a backup and restore mechanism in case of any mishaps.  We are looking for a full backup of the Jenkins and so decided to go with PeriodicBackup plugin.  With this plugin, the backup works fine.  But, we are trying to restore it in a completely new Jenkins setup and not able to restore.  When we pick the backup file that we want to restore from, it says restoring, but nothing happens.  Even after waiting out for over 30 mins, nothing is restored.  As a work around, I stopped the Jenkins service, renamed the 'JENKINS_HOME' (/var/lib/jenkins) folder to 'jenkins_backup' and created another folder called 'jenkins' in the same location.  In this folder, I copied all the files from the backup and then started Jenkins (NOTE: I didn't change the .owner file.  I'm using the .owner file of the new Jenkins only).  Jenkins came up and I was able to see all the configurations from the old one.  Everything was fine.  I let it like that overnight and in the morning when I came and tried to access it, Jenkins was way too slow and I was not able to save any new configuration.  When I try I'm getting 
java.io.ioexception: too many open files jenkins
error.  I restarted the jenkins and it's taking too long for the UI to come up.  Am I doing anything wrong here?  Any suggestions for my requirement?

P.S. - I tried SCMSync plugin and ThinBackup.  But, problem with these are that they don't do full backup.  But, with ThinBackup both backup and restore works fine.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CAKotrh3N0yO5H-n7fcGf%3D%3DZJQyh76Qhag_ZcVjkw3mNrFuJafQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Jenkins Backup and Restore

Artur Szostak
In reply to this post by gotviseryon
Just my opinion: but I would forget about the backup plugins in Jenkins for full production backups. They all do a poor job when it comes to a full backup for a production system that also needs high availability. The best that can be done at the moment is to run Jenkins on a file system or data store that is able to perform copy-on-write snapshots. From within Jenkins you will want to setup a simple job that triggers the copy-on-write snapshot when no other job is running. Then use your normal production backup solutions to backup the snapshot in the background.

Cheers

Artur

________________________________________
From: [hidden email] <[hidden email]> on behalf of [hidden email] <[hidden email]>
Sent: 01 November 2018 21:36
To: Jenkins Users
Subject: Jenkins Backup and Restore

We have a Jenkins instance running and we are trying to setup a backup and restore mechanism in case of any mishaps.  We are looking for a full backup of the Jenkins and so decided to go with PeriodicBackup plugin.  With this plugin, the backup works fine.  But, we are trying to restore it in a completely new Jenkins setup and not able to restore.  When we pick the backup file that we want to restore from, it says restoring, but nothing happens.  Even after waiting out for over 30 mins, nothing is restored.  As a work around, I stopped the Jenkins service, renamed the 'JENKINS_HOME' (/var/lib/jenkins) folder to 'jenkins_backup' and created another folder called 'jenkins' in the same location.  In this folder, I copied all the files from the backup and then started Jenkins (NOTE: I didn't change the .owner file.  I'm using the .owner file of the new Jenkins only).  Jenkins came up and I was able to see all the configurations from the old one.  Everything was fine.  I let it like that overnight and in the morning when I came and tried to access it, Jenkins was way too slow and I was not able to save any new configuration.  When I try I'm getting
java.io.ioexception: too many open files jenkins
error.  I restarted the jenkins and it's taking too long for the UI to come up.  Am I doing anything wrong here?  Any suggestions for my requirement?

P.S. - I tried SCMSync plugin and ThinBackup.  But, problem with these are that they don't do full backup.  But, with ThinBackup both backup and restore works fine.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email]<mailto:[hidden email]>.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com<https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com?utm_medium=email&utm_source=footer>.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/c318b5c6cfce4024ab5d859fac9fe1f3%40partner.eso.org.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Jenkins Backup and Restore

jmellor
I have had restore issues with *ALL* of the backup plugins that I have
tried.  The problem is that they are using stupid mechanisms to copy
the filesystem that cannot correctly copy a symlink.  This breaks the
LastSuccessfulBuild and LastUnsuccessfulBuild and LastFailedBuild
symlinks, causing lots of secondary issues.

Doing my own using cpio or rsync in a cronjob only messes up copying
very large files that exceed the archive limits (4GB for cpio for
instance).


On Mon, 2018-11-26 at 21:12 +0000, Artur Szostak wrote:

> Just my opinion: but I would forget about the backup plugins in
> Jenkins for full production backups. They all do a poor job when it
> comes to a full backup for a production system that also needs high
> availability. The best that can be done at the moment is to run
> Jenkins on a file system or data store that is able to perform copy-
> on-write snapshots. From within Jenkins you will want to setup a
> simple job that triggers the copy-on-write snapshot when no other job
> is running. Then use your normal production backup solutions to
> backup the snapshot in the background.
>
> Cheers
>
> Artur
>
> ________________________________________
> From: [hidden email] <
> [hidden email]> on behalf of [hidden email]
> <[hidden email]>
> Sent: 01 November 2018 21:36
> To: Jenkins Users
> Subject: Jenkins Backup and Restore
>
> We have a Jenkins instance running and we are trying to setup a
> backup and restore mechanism in case of any mishaps.  We are looking
> for a full backup of the Jenkins and so decided to go with
> PeriodicBackup plugin.  With this plugin, the backup works
> fine.  But, we are trying to restore it in a completely new Jenkins
> setup and not able to restore.  When we pick the backup file that we
> want to restore from, it says restoring, but nothing happens.  Even
> after waiting out for over 30 mins, nothing is restored.  As a work
> around, I stopped the Jenkins service, renamed the 'JENKINS_HOME'
> (/var/lib/jenkins) folder to 'jenkins_backup' and created another
> folder called 'jenkins' in the same location.  In this folder, I
> copied all the files from the backup and then started Jenkins (NOTE:
> I didn't change the .owner file.  I'm using the .owner file of the
> new Jenkins only).  Jenkins came up and I was able to see all the
> configurations from the old one.  Everything was fine.  I let it like
> that overnight and in the morning when I came and tried to access it,
> Jenkins was way too slow and I was not able to save any new
> configuration.  When I try I'm getting
> java.io.ioexception: too many open files jenkins
> error.  I restarted the jenkins and it's taking too long for the UI
> to come up.  Am I doing anything wrong here?  Any suggestions for my
> requirement?
>
> P.S. - I tried SCMSync plugin and ThinBackup.  But, problem with
> these are that they don't do full backup.  But, with ThinBackup both
> backup and restore works fine.
>
> --
> You received this message because you are subscribed to the Google
> Groups "Jenkins Users" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to [hidden email]<mailto:
> [hidden email]>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com<https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com?utm_medium=email&utm_source=footer>
> ;.
> For more options, visit https://groups.google.com/d/optout.
>

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/0e5a17f5dc808937b44f25f634cc6eeb76fb907a.camel%40esentire.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Jenkins Backup and Restore

gotviseryon
I wish for a tool like Jenkins it had a backup and restore option built in.

On Monday, November 26, 2018 at 4:35:04 PM UTC-5, john.mellor wrote:
I have had restore issues with *ALL* of the backup plugins that I have
tried.  The problem is that they are using stupid mechanisms to copy
the filesystem that cannot correctly copy a symlink.  This breaks the
LastSuccessfulBuild and LastUnsuccessfulBuild and LastFailedBuild
symlinks, causing lots of secondary issues.

Doing my own using cpio or rsync in a cronjob only messes up copying
very large files that exceed the archive limits (4GB for cpio for
instance).


On Mon, 2018-11-26 at 21:12 +0000, Artur Szostak wrote:

> Just my opinion: but I would forget about the backup plugins in
> Jenkins for full production backups. They all do a poor job when it
> comes to a full backup for a production system that also needs high
> availability. The best that can be done at the moment is to run
> Jenkins on a file system or data store that is able to perform copy-
> on-write snapshots. From within Jenkins you will want to setup a
> simple job that triggers the copy-on-write snapshot when no other job
> is running. Then use your normal production backup solutions to
> backup the snapshot in the background.
>
> Cheers
>
> Artur
>
> ________________________________________
> From: <a href="javascript:" target="_blank" gdf-obfuscated-mailto="I8qEZuBpBAAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">jenkins...@googlegroups.com <
> <a href="javascript:" target="_blank" gdf-obfuscated-mailto="I8qEZuBpBAAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">jenkins...@googlegroups.com> on behalf of <a href="javascript:" target="_blank" gdf-obfuscated-mailto="I8qEZuBpBAAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">gotvi...@...
> <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="I8qEZuBpBAAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">gotvi...@...>
> Sent: 01 November 2018 21:36
> To: Jenkins Users
> Subject: Jenkins Backup and Restore
>
> We have a Jenkins instance running and we are trying to setup a
> backup and restore mechanism in case of any mishaps.  We are looking
> for a full backup of the Jenkins and so decided to go with
> PeriodicBackup plugin.  With this plugin, the backup works
> fine.  But, we are trying to restore it in a completely new Jenkins
> setup and not able to restore.  When we pick the backup file that we
> want to restore from, it says restoring, but nothing happens.  Even
> after waiting out for over 30 mins, nothing is restored.  As a work
> around, I stopped the Jenkins service, renamed the 'JENKINS_HOME'
> (/var/lib/jenkins) folder to 'jenkins_backup' and created another
> folder called 'jenkins' in the same location.  In this folder, I
> copied all the files from the backup and then started Jenkins (NOTE:
> I didn't change the .owner file.  I'm using the .owner file of the
> new Jenkins only).  Jenkins came up and I was able to see all the
> configurations from the old one.  Everything was fine.  I let it like
> that overnight and in the morning when I came and tried to access it,
> Jenkins was way too slow and I was not able to save any new
> configuration.  When I try I'm getting
> java.io.ioexception: too many open files jenkins
> error.  I restarted the jenkins and it's taking too long for the UI
> to come up.  Am I doing anything wrong here?  Any suggestions for my
> requirement?
>
> P.S. - I tried SCMSync plugin and ThinBackup.  But, problem with
> these are that they don't do full backup.  But, with ThinBackup both
> backup and restore works fine.
>
> --
> You received this message because you are subscribed to the Google
> Groups "Jenkins Users" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="I8qEZuBpBAAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">jenkinsci-use...@googlegroups.com<mailto:
> <a href="javascript:" target="_blank" gdf-obfuscated-mailto="I8qEZuBpBAAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">jenkinsci-use...@googlegroups.com>.
> To view this discussion on the web visit
> <a href="https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com&#39;;return true;">https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com<<a href="https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com?utm_medium=email&amp;utm_source=footer" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;">https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com?utm_medium=email&utm_source=footer>
> ;.
> For more options, visit <a href="https://groups.google.com/d/optout" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;">https://groups.google.com/d/optout.
>

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/3501a70d-1b2f-4163-88e8-a2d497f64d43%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Jenkins Backup and Restore

gotviseryon
In reply to this post by Artur Szostak
Our cloud is Azure.  Do you have any suggestions for this?

On Monday, November 26, 2018 at 4:13:06 PM UTC-5, Artur Szostak wrote:
Just my opinion: but I would forget about the backup plugins in Jenkins for full production backups. They all do a poor job when it comes to a full backup for a production system that also needs high availability. The best that can be done at the moment is to run Jenkins on a file system or data store that is able to perform copy-on-write snapshots. From within Jenkins you will want to setup a simple job that triggers the copy-on-write snapshot when no other job is running. Then use your normal production backup solutions to backup the snapshot in the background.

Cheers

Artur

________________________________________
From: <a href="javascript:" target="_blank" gdf-obfuscated-mailto="U4XXga1oBAAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">jenkins...@googlegroups.com <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="U4XXga1oBAAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">jenkins...@googlegroups.com> on behalf of <a href="javascript:" target="_blank" gdf-obfuscated-mailto="U4XXga1oBAAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">gotvi...@... <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="U4XXga1oBAAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">gotvi...@...>
Sent: 01 November 2018 21:36
To: Jenkins Users
Subject: Jenkins Backup and Restore

We have a Jenkins instance running and we are trying to setup a backup and restore mechanism in case of any mishaps.  We are looking for a full backup of the Jenkins and so decided to go with PeriodicBackup plugin.  With this plugin, the backup works fine.  But, we are trying to restore it in a completely new Jenkins setup and not able to restore.  When we pick the backup file that we want to restore from, it says restoring, but nothing happens.  Even after waiting out for over 30 mins, nothing is restored.  As a work around, I stopped the Jenkins service, renamed the 'JENKINS_HOME' (/var/lib/jenkins) folder to 'jenkins_backup' and created another folder called 'jenkins' in the same location.  In this folder, I copied all the files from the backup and then started Jenkins (NOTE: I didn't change the .owner file.  I'm using the .owner file of the new Jenkins only).  Jenkins came up and I was able to see all the configurations from the old one.  Everything was fine.  I let it like that overnight and in the morning when I came and tried to access it, Jenkins was way too slow and I was not able to save any new configuration.  When I try I'm getting
java.io.ioexception: too many open files jenkins
error.  I restarted the jenkins and it's taking too long for the UI to come up.  Am I doing anything wrong here?  Any suggestions for my requirement?

P.S. - I tried SCMSync plugin and ThinBackup.  But, problem with these are that they don't do full backup.  But, with ThinBackup both backup and restore works fine.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="U4XXga1oBAAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">jenkinsci-use...@googlegroups.com<mailto:<a href="javascript:" target="_blank" gdf-obfuscated-mailto="U4XXga1oBAAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">jenkinsci-users+unsubscribe@googlegroups.com>.
To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com&#39;;return true;">https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com<<a href="https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com?utm_medium=email&amp;utm_source=footer" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;">https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com?utm_medium=email&utm_source=footer>.
For more options, visit <a href="https://groups.google.com/d/optout" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;">https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/7c46683a-fbfb-4d79-b470-10246e8999d6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Jenkins Backup and Restore

jxpearce
In reply to this post by jmellor

Have you considered sending a PR to (or at least filing a Github issue for) the plugins in question?

 

It's probably an easy fix to handle symlinks

 

Best

Jeff


On Monday, November 26, 2018 at 1:35:04 PM UTC-8, john.mellor wrote:
I have had restore issues with *ALL* of the backup plugins that I have
tried.  The problem is that they are using stupid mechanisms to copy
the filesystem that cannot correctly copy a symlink.  This breaks the
LastSuccessfulBuild and LastUnsuccessfulBuild and LastFailedBuild
symlinks, causing lots of secondary issues.

Doing my own using cpio or rsync in a cronjob only messes up copying
very large files that exceed the archive limits (4GB for cpio for
instance).


On Mon, 2018-11-26 at 21:12 +0000, Artur Szostak wrote:

> Just my opinion: but I would forget about the backup plugins in
> Jenkins for full production backups. They all do a poor job when it
> comes to a full backup for a production system that also needs high
> availability. The best that can be done at the moment is to run
> Jenkins on a file system or data store that is able to perform copy-
> on-write snapshots. From within Jenkins you will want to setup a
> simple job that triggers the copy-on-write snapshot when no other job
> is running. Then use your normal production backup solutions to
> backup the snapshot in the background.
>
> Cheers
>
> Artur
>
> ________________________________________
> From: <a href="javascript:" target="_blank" gdf-obfuscated-mailto="I8qEZuBpBAAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">jenkins...@googlegroups.com <
> <a href="javascript:" target="_blank" gdf-obfuscated-mailto="I8qEZuBpBAAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">jenkins...@googlegroups.com> on behalf of <a href="javascript:" target="_blank" gdf-obfuscated-mailto="I8qEZuBpBAAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">gotvi...@...
> <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="I8qEZuBpBAAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">gotvi...@...>
> Sent: 01 November 2018 21:36
> To: Jenkins Users
> Subject: Jenkins Backup and Restore
>
> We have a Jenkins instance running and we are trying to setup a
> backup and restore mechanism in case of any mishaps.  We are looking
> for a full backup of the Jenkins and so decided to go with
> PeriodicBackup plugin.  With this plugin, the backup works
> fine.  But, we are trying to restore it in a completely new Jenkins
> setup and not able to restore.  When we pick the backup file that we
> want to restore from, it says restoring, but nothing happens.  Even
> after waiting out for over 30 mins, nothing is restored.  As a work
> around, I stopped the Jenkins service, renamed the 'JENKINS_HOME'
> (/var/lib/jenkins) folder to 'jenkins_backup' and created another
> folder called 'jenkins' in the same location.  In this folder, I
> copied all the files from the backup and then started Jenkins (NOTE:
> I didn't change the .owner file.  I'm using the .owner file of the
> new Jenkins only).  Jenkins came up and I was able to see all the
> configurations from the old one.  Everything was fine.  I let it like
> that overnight and in the morning when I came and tried to access it,
> Jenkins was way too slow and I was not able to save any new
> configuration.  When I try I'm getting
> java.io.ioexception: too many open files jenkins
> error.  I restarted the jenkins and it's taking too long for the UI
> to come up.  Am I doing anything wrong here?  Any suggestions for my
> requirement?
>
> P.S. - I tried SCMSync plugin and ThinBackup.  But, problem with
> these are that they don't do full backup.  But, with ThinBackup both
> backup and restore works fine.
>
> --
> You received this message because you are subscribed to the Google
> Groups "Jenkins Users" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="I8qEZuBpBAAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">jenkinsci-use...@googlegroups.com<mailto:
> <a href="javascript:" target="_blank" gdf-obfuscated-mailto="I8qEZuBpBAAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">jenkinsci-use...@googlegroups.com>.
> To view this discussion on the web visit
> <a href="https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com&#39;;return true;">https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com<<a href="https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com?utm_medium=email&amp;utm_source=footer" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;">https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com?utm_medium=email&utm_source=footer>
> ;.
> For more options, visit <a href="https://groups.google.com/d/optout" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;">https://groups.google.com/d/optout.
>

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/e16c5852-8d61-4dab-b717-81e3b6e7849e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Jenkins Backup and Restore

jxpearce
In reply to this post by gotviseryon
The beauty of Jenkins is that it's NOT built-in, but can be easily added as a plugin (of which there are several).

On Monday, November 26, 2018 at 1:41:19 PM UTC-8, [hidden email] wrote:
I wish for a tool like Jenkins it had a backup and restore option built in.

On Monday, November 26, 2018 at 4:35:04 PM UTC-5, john.mellor wrote:
I have had restore issues with *ALL* of the backup plugins that I have
tried.  The problem is that they are using stupid mechanisms to copy
the filesystem that cannot correctly copy a symlink.  This breaks the
LastSuccessfulBuild and LastUnsuccessfulBuild and LastFailedBuild
symlinks, causing lots of secondary issues.

Doing my own using cpio or rsync in a cronjob only messes up copying
very large files that exceed the archive limits (4GB for cpio for
instance).


On Mon, 2018-11-26 at 21:12 +0000, Artur Szostak wrote:

> Just my opinion: but I would forget about the backup plugins in
> Jenkins for full production backups. They all do a poor job when it
> comes to a full backup for a production system that also needs high
> availability. The best that can be done at the moment is to run
> Jenkins on a file system or data store that is able to perform copy-
> on-write snapshots. From within Jenkins you will want to setup a
> simple job that triggers the copy-on-write snapshot when no other job
> is running. Then use your normal production backup solutions to
> backup the snapshot in the background.
>
> Cheers
>
> Artur
>
> ________________________________________
> From: [hidden email] <
> [hidden email]> on behalf of [hidden email]
> <[hidden email]>
> Sent: 01 November 2018 21:36
> To: Jenkins Users
> Subject: Jenkins Backup and Restore
>
> We have a Jenkins instance running and we are trying to setup a
> backup and restore mechanism in case of any mishaps.  We are looking
> for a full backup of the Jenkins and so decided to go with
> PeriodicBackup plugin.  With this plugin, the backup works
> fine.  But, we are trying to restore it in a completely new Jenkins
> setup and not able to restore.  When we pick the backup file that we
> want to restore from, it says restoring, but nothing happens.  Even
> after waiting out for over 30 mins, nothing is restored.  As a work
> around, I stopped the Jenkins service, renamed the 'JENKINS_HOME'
> (/var/lib/jenkins) folder to 'jenkins_backup' and created another
> folder called 'jenkins' in the same location.  In this folder, I
> copied all the files from the backup and then started Jenkins (NOTE:
> I didn't change the .owner file.  I'm using the .owner file of the
> new Jenkins only).  Jenkins came up and I was able to see all the
> configurations from the old one.  Everything was fine.  I let it like
> that overnight and in the morning when I came and tried to access it,
> Jenkins was way too slow and I was not able to save any new
> configuration.  When I try I'm getting
> java.io.ioexception: too many open files jenkins
> error.  I restarted the jenkins and it's taking too long for the UI
> to come up.  Am I doing anything wrong here?  Any suggestions for my
> requirement?
>
> P.S. - I tried SCMSync plugin and ThinBackup.  But, problem with
> these are that they don't do full backup.  But, with ThinBackup both
> backup and restore works fine.
>
> --
> You received this message because you are subscribed to the Google
> Groups "Jenkins Users" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to jenkinsci-use...@googlegroups.com<mailto:
> jenkinsci-use...@googlegroups.com>.
> To view this discussion on the web visit
> <a href="https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com&#39;;return true;">https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com<<a href="https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com?utm_medium=email&amp;utm_source=footer" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;">https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com?utm_medium=email&utm_source=footer>
> ;.
> For more options, visit <a href="https://groups.google.com/d/optout" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;">https://groups.google.com/d/optout.
>

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/068329c2-bd4c-468b-ac60-f7c138e18b37%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Jenkins Backup and Restore

RAJENDRA PRASAD
In reply to this post by gotviseryon
Hi ,
I have another solution, just document the list of plugins in old Jenkins server and instead of backup and restore plugins just reinstall plugins on new Jenkins server.

Thanks,
Rajendra

On Fri, 2 Nov, 2018, 02:06 <[hidden email] wrote:
We have a Jenkins instance running and we are trying to setup a backup and restore mechanism in case of any mishaps.  We are looking for a full backup of the Jenkins and so decided to go with PeriodicBackup plugin.  With this plugin, the backup works fine.  But, we are trying to restore it in a completely new Jenkins setup and not able to restore.  When we pick the backup file that we want to restore from, it says restoring, but nothing happens.  Even after waiting out for over 30 mins, nothing is restored.  As a work around, I stopped the Jenkins service, renamed the 'JENKINS_HOME' (/var/lib/jenkins) folder to 'jenkins_backup' and created another folder called 'jenkins' in the same location.  In this folder, I copied all the files from the backup and then started Jenkins (NOTE: I didn't change the .owner file.  I'm using the .owner file of the new Jenkins only).  Jenkins came up and I was able to see all the configurations from the old one.  Everything was fine.  I let it like that overnight and in the morning when I came and tried to access it, Jenkins was way too slow and I was not able to save any new configuration.  When I try I'm getting 
java.io.ioexception: too many open files jenkins
error.  I restarted the jenkins and it's taking too long for the UI to come up.  Am I doing anything wrong here?  Any suggestions for my requirement?

P.S. - I tried SCMSync plugin and ThinBackup.  But, problem with these are that they don't do full backup.  But, with ThinBackup both backup and restore works fine.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CAMrg02T5QPm9WRN2dpOOEtTZUnjtiqXAEONDVt1ahn4nh-ZM%2Bg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Jenkins Backup and Restore

gotviseryon
In reply to this post by jxpearce
The plugin team's been unresponsive for ages.  No help there.

On Monday, November 26, 2018 at 4:49:17 PM UTC-5, [hidden email] wrote:

Have you considered sending a PR to (or at least filing a Github issue for) the plugins in question?

 

It's probably an easy fix to handle symlinks

 

Best

Jeff


On Monday, November 26, 2018 at 1:35:04 PM UTC-8, john.mellor wrote:
I have had restore issues with *ALL* of the backup plugins that I have
tried.  The problem is that they are using stupid mechanisms to copy
the filesystem that cannot correctly copy a symlink.  This breaks the
LastSuccessfulBuild and LastUnsuccessfulBuild and LastFailedBuild
symlinks, causing lots of secondary issues.

Doing my own using cpio or rsync in a cronjob only messes up copying
very large files that exceed the archive limits (4GB for cpio for
instance).


On Mon, 2018-11-26 at 21:12 +0000, Artur Szostak wrote:

> Just my opinion: but I would forget about the backup plugins in
> Jenkins for full production backups. They all do a poor job when it
> comes to a full backup for a production system that also needs high
> availability. The best that can be done at the moment is to run
> Jenkins on a file system or data store that is able to perform copy-
> on-write snapshots. From within Jenkins you will want to setup a
> simple job that triggers the copy-on-write snapshot when no other job
> is running. Then use your normal production backup solutions to
> backup the snapshot in the background.
>
> Cheers
>
> Artur
>
> ________________________________________
> From: [hidden email] <
> [hidden email]> on behalf of [hidden email]
> <[hidden email]>
> Sent: 01 November 2018 21:36
> To: Jenkins Users
> Subject: Jenkins Backup and Restore
>
> We have a Jenkins instance running and we are trying to setup a
> backup and restore mechanism in case of any mishaps.  We are looking
> for a full backup of the Jenkins and so decided to go with
> PeriodicBackup plugin.  With this plugin, the backup works
> fine.  But, we are trying to restore it in a completely new Jenkins
> setup and not able to restore.  When we pick the backup file that we
> want to restore from, it says restoring, but nothing happens.  Even
> after waiting out for over 30 mins, nothing is restored.  As a work
> around, I stopped the Jenkins service, renamed the 'JENKINS_HOME'
> (/var/lib/jenkins) folder to 'jenkins_backup' and created another
> folder called 'jenkins' in the same location.  In this folder, I
> copied all the files from the backup and then started Jenkins (NOTE:
> I didn't change the .owner file.  I'm using the .owner file of the
> new Jenkins only).  Jenkins came up and I was able to see all the
> configurations from the old one.  Everything was fine.  I let it like
> that overnight and in the morning when I came and tried to access it,
> Jenkins was way too slow and I was not able to save any new
> configuration.  When I try I'm getting
> java.io.ioexception: too many open files jenkins
> error.  I restarted the jenkins and it's taking too long for the UI
> to come up.  Am I doing anything wrong here?  Any suggestions for my
> requirement?
>
> P.S. - I tried SCMSync plugin and ThinBackup.  But, problem with
> these are that they don't do full backup.  But, with ThinBackup both
> backup and restore works fine.
>
> --
> You received this message because you are subscribed to the Google
> Groups "Jenkins Users" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to jenkinsci-use...@googlegroups.com<mailto:
> jenkinsci-use...@googlegroups.com>.
> To view this discussion on the web visit
> <a href="https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com&#39;;return true;">https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com<<a href="https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com?utm_medium=email&amp;utm_source=footer" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;">https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com?utm_medium=email&utm_source=footer>
> ;.
> For more options, visit <a href="https://groups.google.com/d/optout" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;">https://groups.google.com/d/optout.
>

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/f4851afe-87a4-4a8c-8499-9d65388e8cad%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Jenkins Backup and Restore

gotviseryon
In reply to this post by jxpearce
Tried out a bunch of them.  Unfortunately, none of them are good for a production scale.

On Monday, November 26, 2018 at 5:18:04 PM UTC-5, [hidden email] wrote:
The beauty of Jenkins is that it's NOT built-in, but can be easily added as a plugin (of which there are several).

On Monday, November 26, 2018 at 1:41:19 PM UTC-8, [hidden email] wrote:
I wish for a tool like Jenkins it had a backup and restore option built in.

On Monday, November 26, 2018 at 4:35:04 PM UTC-5, john.mellor wrote:
I have had restore issues with *ALL* of the backup plugins that I have
tried.  The problem is that they are using stupid mechanisms to copy
the filesystem that cannot correctly copy a symlink.  This breaks the
LastSuccessfulBuild and LastUnsuccessfulBuild and LastFailedBuild
symlinks, causing lots of secondary issues.

Doing my own using cpio or rsync in a cronjob only messes up copying
very large files that exceed the archive limits (4GB for cpio for
instance).


On Mon, 2018-11-26 at 21:12 +0000, Artur Szostak wrote:

> Just my opinion: but I would forget about the backup plugins in
> Jenkins for full production backups. They all do a poor job when it
> comes to a full backup for a production system that also needs high
> availability. The best that can be done at the moment is to run
> Jenkins on a file system or data store that is able to perform copy-
> on-write snapshots. From within Jenkins you will want to setup a
> simple job that triggers the copy-on-write snapshot when no other job
> is running. Then use your normal production backup solutions to
> backup the snapshot in the background.
>
> Cheers
>
> Artur
>
> ________________________________________
> From: [hidden email] <
> [hidden email]> on behalf of [hidden email]
> <[hidden email]>
> Sent: 01 November 2018 21:36
> To: Jenkins Users
> Subject: Jenkins Backup and Restore
>
> We have a Jenkins instance running and we are trying to setup a
> backup and restore mechanism in case of any mishaps.  We are looking
> for a full backup of the Jenkins and so decided to go with
> PeriodicBackup plugin.  With this plugin, the backup works
> fine.  But, we are trying to restore it in a completely new Jenkins
> setup and not able to restore.  When we pick the backup file that we
> want to restore from, it says restoring, but nothing happens.  Even
> after waiting out for over 30 mins, nothing is restored.  As a work
> around, I stopped the Jenkins service, renamed the 'JENKINS_HOME'
> (/var/lib/jenkins) folder to 'jenkins_backup' and created another
> folder called 'jenkins' in the same location.  In this folder, I
> copied all the files from the backup and then started Jenkins (NOTE:
> I didn't change the .owner file.  I'm using the .owner file of the
> new Jenkins only).  Jenkins came up and I was able to see all the
> configurations from the old one.  Everything was fine.  I let it like
> that overnight and in the morning when I came and tried to access it,
> Jenkins was way too slow and I was not able to save any new
> configuration.  When I try I'm getting
> java.io.ioexception: too many open files jenkins
> error.  I restarted the jenkins and it's taking too long for the UI
> to come up.  Am I doing anything wrong here?  Any suggestions for my
> requirement?
>
> P.S. - I tried SCMSync plugin and ThinBackup.  But, problem with
> these are that they don't do full backup.  But, with ThinBackup both
> backup and restore works fine.
>
> --
> You received this message because you are subscribed to the Google
> Groups "Jenkins Users" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to jenkinsci-use...@googlegroups.com<mailto:
> jenkinsci-use...@googlegroups.com>.
> To view this discussion on the web visit
> <a href="https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com&#39;;return true;">https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com<<a href="https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com?utm_medium=email&amp;utm_source=footer" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;">https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com?utm_medium=email&utm_source=footer>
> ;.
> For more options, visit <a href="https://groups.google.com/d/optout" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;">https://groups.google.com/d/optout.
>

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/f5625308-1145-4d77-b021-62be8a779592%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Jenkins Backup and Restore

gotviseryon
In reply to this post by RAJENDRA PRASAD
I'm not sure what you are trying to say?

On Monday, November 26, 2018 at 8:03:34 PM UTC-5, rajendraprasad reddy wrote:
Hi ,
I have another solution, just document the list of plugins in old Jenkins server and instead of backup and restore plugins just reinstall plugins on new Jenkins server.

Thanks,
Rajendra

On Fri, 2 Nov, 2018, 02:06 <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="wvqrOUF1BAAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">gotvi...@... wrote:
We have a Jenkins instance running and we are trying to setup a backup and restore mechanism in case of any mishaps.  We are looking for a full backup of the Jenkins and so decided to go with PeriodicBackup plugin.  With this plugin, the backup works fine.  But, we are trying to restore it in a completely new Jenkins setup and not able to restore.  When we pick the backup file that we want to restore from, it says restoring, but nothing happens.  Even after waiting out for over 30 mins, nothing is restored.  As a work around, I stopped the Jenkins service, renamed the 'JENKINS_HOME' (/var/lib/jenkins) folder to 'jenkins_backup' and created another folder called 'jenkins' in the same location.  In this folder, I copied all the files from the backup and then started Jenkins (NOTE: I didn't change the .owner file.  I'm using the .owner file of the new Jenkins only).  Jenkins came up and I was able to see all the configurations from the old one.  Everything was fine.  I let it like that overnight and in the morning when I came and tried to access it, Jenkins was way too slow and I was not able to save any new configuration.  When I try I'm getting 
java.io.ioexception: too many open files jenkins
error.  I restarted the jenkins and it's taking too long for the UI to come up.  Am I doing anything wrong here?  Any suggestions for my requirement?

P.S. - I tried SCMSync plugin and ThinBackup.  But, problem with these are that they don't do full backup.  But, with ThinBackup both backup and restore works fine.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="javascript:" rel="nofollow" target="_blank" gdf-obfuscated-mailto="wvqrOUF1BAAJ" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">jenkinsci-use...@googlegroups.com.
To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com?utm_medium=email&amp;utm_source=footer" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;">https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com.
For more options, visit <a href="https://groups.google.com/d/optout" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;">https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/9d3fad8e-7bbe-4472-86dd-57ef3d0da960%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Jenkins Backup and Restore

jxpearce
In reply to this post by gotviseryon

What issues are you seeing that prevent them from being good for production scale?

 

From: <[hidden email]> on behalf of "[hidden email]" <[hidden email]>
Reply-To: "[hidden email]" <[hidden email]>
Date: Tuesday, November 27, 2018 at 6:49 AM
To: Jenkins Users <[hidden email]>
Subject: Re: Jenkins Backup and Restore

 

Tried out a bunch of them.  Unfortunately, none of them are good for a production scale.

On Monday, November 26, 2018 at 5:18:04 PM UTC-5, [hidden email] wrote:

The beauty of Jenkins is that it's NOT built-in, but can be easily added as a plugin (of which there are several).

On Monday, November 26, 2018 at 1:41:19 PM UTC-8, [hidden email] wrote:

I wish for a tool like Jenkins it had a backup and restore option built in.

On Monday, November 26, 2018 at 4:35:04 PM UTC-5, john.mellor wrote:

I have had restore issues with *ALL* of the backup plugins that I have
tried.  The problem is that they are using stupid mechanisms to copy
the filesystem that cannot correctly copy a symlink.  This breaks the
LastSuccessfulBuild and LastUnsuccessfulBuild and LastFailedBuild
symlinks, causing lots of secondary issues.

Doing my own using cpio or rsync in a cronjob only messes up copying
very large files that exceed the archive limits (4GB for cpio for
instance).


On Mon, 2018-11-26 at 21:12 +0000, Artur Szostak wrote:
> Just my opinion: but I would forget about the backup plugins in
> Jenkins for full production backups. They all do a poor job when it
> comes to a full backup for a production system that also needs high
> availability. The best that can be done at the moment is to run
> Jenkins on a file system or data store that is able to perform copy-
> on-write snapshots. From within Jenkins you will want to setup a
> simple job that triggers the copy-on-write snapshot when no other job
> is running. Then use your normal production backup solutions to
> backup the snapshot in the background.
>
> Cheers
>
> Artur
>
> ________________________________________
> From: [hidden email] <
> [hidden email]> on behalf of [hidden email]
> <[hidden email]>
> Sent: 01 November 2018 21:36
> To: Jenkins Users
> Subject: Jenkins Backup and Restore
>
> We have a Jenkins instance running and we are trying to setup a
> backup and restore mechanism in case of any mishaps.  We are looking
> for a full backup of the Jenkins and so decided to go with
> PeriodicBackup plugin.  With this plugin, the backup works
> fine.  But, we are trying to restore it in a completely new Jenkins
> setup and not able to restore.  When we pick the backup file that we
> want to restore from, it says restoring, but nothing happens.  Even
> after waiting out for over 30 mins, nothing is restored.  As a work
> around, I stopped the Jenkins service, renamed the 'JENKINS_HOME'
> (/var/lib/jenkins) folder to 'jenkins_backup' and created another
> folder called 'jenkins' in the same location.  In this folder, I
> copied all the files from the backup and then started Jenkins (NOTE:
> I didn't change the .owner file.  I'm using the .owner file of the
> new Jenkins only).  Jenkins came up and I was able to see all the
> configurations from the old one.  Everything was fine.  I let it like
> that overnight and in the morning when I came and tried to access it,
> Jenkins was way too slow and I was not able to save any new
> configuration.  When I try I'm getting
> java.io.ioexception: too many open files jenkins
> error.  I restarted the jenkins and it's taking too long for the UI
> to come up.  Am I doing anything wrong here?  Any suggestions for my
> requirement?
>
> P.S. - I tried SCMSync plugin and ThinBackup.  But, problem with
> these are that they don't do full backup.  But, with ThinBackup both
> backup and restore works fine.
>
> --
> You received this message because you are subscribed to the Google
> Groups "Jenkins Users" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to [hidden email]<mailto:
> [hidden email]>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com<https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com?utm_medium=email&utm_source=footer>
> ;.
> For more options, visit https://groups.google.com/d/optout.
>

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/f5625308-1145-4d77-b021-62be8a779592%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/588AA5F3-DCE6-4516-B0C7-7E2C161E3D5C%40godaddy.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Jenkins Backup and Restore

Mark Waite-2
In reply to this post by gotviseryon
He's saying that if you don't need to backup and restore your job definitions (possibly they are all multibranch Pipeline jobs that are easy to recreate, or they are all defined with the Job DSL plugin so they will be automatically recreated or they are defined in a Docker image) and don't need to backup and restore your credential definitions (same reasons) and don't need to backup and restore other configuration settings (possibly because you're using Configuration as Code plugin to store your configuration in a source control system), then there are backup concepts that use a source control system as your backup system.

In your use case, I suspect you want to use a backup.

Mark Waite

On Tue, Nov 27, 2018 at 7:50 AM <[hidden email]> wrote:
I'm not sure what you are trying to say?

On Monday, November 26, 2018 at 8:03:34 PM UTC-5, rajendraprasad reddy wrote:
Hi ,
I have another solution, just document the list of plugins in old Jenkins server and instead of backup and restore plugins just reinstall plugins on new Jenkins server.

Thanks,
Rajendra

On Fri, 2 Nov, 2018, 02:06 <[hidden email] wrote:
We have a Jenkins instance running and we are trying to setup a backup and restore mechanism in case of any mishaps.  We are looking for a full backup of the Jenkins and so decided to go with PeriodicBackup plugin.  With this plugin, the backup works fine.  But, we are trying to restore it in a completely new Jenkins setup and not able to restore.  When we pick the backup file that we want to restore from, it says restoring, but nothing happens.  Even after waiting out for over 30 mins, nothing is restored.  As a work around, I stopped the Jenkins service, renamed the 'JENKINS_HOME' (/var/lib/jenkins) folder to 'jenkins_backup' and created another folder called 'jenkins' in the same location.  In this folder, I copied all the files from the backup and then started Jenkins (NOTE: I didn't change the .owner file.  I'm using the .owner file of the new Jenkins only).  Jenkins came up and I was able to see all the configurations from the old one.  Everything was fine.  I let it like that overnight and in the morning when I came and tried to access it, Jenkins was way too slow and I was not able to save any new configuration.  When I try I'm getting 
java.io.ioexception: too many open files jenkins
error.  I restarted the jenkins and it's taking too long for the UI to come up.  Am I doing anything wrong here?  Any suggestions for my requirement?

P.S. - I tried SCMSync plugin and ThinBackup.  But, problem with these are that they don't do full backup.  But, with ThinBackup both backup and restore works fine.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/9d3fad8e-7bbe-4472-86dd-57ef3d0da960%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


--
Thanks!
Mark Waite

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CAO49JtH-ydL8JkmnFurJ06s%2BEn-b%3DWaNmb_KT%3DBFiz6mbZK8QQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Jenkins Backup and Restore

gotviseryon
In reply to this post by jxpearce
We are looking for a full backup of the JENKINS_HOME folder (automated).  We looked at SMC Sync plugin, ThinBackup, Backup and PeriodicBackup.  PeriodicBackup was the only one which offered both.  But, with this we are facing the following issues

1) The values that we set for "# of backups" and the "# of days to store backup" does not work.  It keeps creating more and more without deleting the previous ones.
2) The restore feature does not work.  I took one backup and tried to restore a new Jenkins instance.  It says "restoring backup" in the UI, but no matter how long we wait, it doesn't restore.  I had to manually extract the file to the JENKINS_HOME folder.

On Tuesday, November 27, 2018 at 12:25:06 PM UTC-5, Jeff Pearce wrote:

What issues are you seeing that prevent them from being good for production scale?

 

From: <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="SdlC-NCqBAAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">jenkins...@googlegroups.com> on behalf of "<a href="javascript:" target="_blank" gdf-obfuscated-mailto="SdlC-NCqBAAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">gotvi...@..." <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="SdlC-NCqBAAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">gotvi...@...>
Reply-To: "<a href="javascript:" target="_blank" gdf-obfuscated-mailto="SdlC-NCqBAAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">jenkins...@googlegroups.com" <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="SdlC-NCqBAAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">jenkins...@googlegroups.com>
Date: Tuesday, November 27, 2018 at 6:49 AM
To: Jenkins Users <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="SdlC-NCqBAAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">jenkins...@googlegroups.com>
Subject: Re: Jenkins Backup and Restore

 

Tried out a bunch of them.  Unfortunately, none of them are good for a production scale.

On Monday, November 26, 2018 at 5:18:04 PM UTC-5, [hidden email] wrote:

The beauty of Jenkins is that it's NOT built-in, but can be easily added as a plugin (of which there are several).

On Monday, November 26, 2018 at 1:41:19 PM UTC-8, [hidden email] wrote:

I wish for a tool like Jenkins it had a backup and restore option built in.

On Monday, November 26, 2018 at 4:35:04 PM UTC-5, john.mellor wrote:

I have had restore issues with *ALL* of the backup plugins that I have
tried.  The problem is that they are using stupid mechanisms to copy
the filesystem that cannot correctly copy a symlink.  This breaks the
LastSuccessfulBuild and LastUnsuccessfulBuild and LastFailedBuild
symlinks, causing lots of secondary issues.

Doing my own using cpio or rsync in a cronjob only messes up copying
very large files that exceed the archive limits (4GB for cpio for
instance).


On Mon, 2018-11-26 at 21:12 +0000, Artur Szostak wrote:
> Just my opinion: but I would forget about the backup plugins in
> Jenkins for full production backups. They all do a poor job when it
> comes to a full backup for a production system that also needs high
> availability. The best that can be done at the moment is to run
> Jenkins on a file system or data store that is able to perform copy-
> on-write snapshots. From within Jenkins you will want to setup a
> simple job that triggers the copy-on-write snapshot when no other job
> is running. Then use your normal production backup solutions to
> backup the snapshot in the background.
>
> Cheers
>
> Artur
>
> ________________________________________
> From: [hidden email] <
> [hidden email]> on behalf of [hidden email]
> <[hidden email]>
> Sent: 01 November 2018 21:36
> To: Jenkins Users
> Subject: Jenkins Backup and Restore
>
> We have a Jenkins instance running and we are trying to setup a
> backup and restore mechanism in case of any mishaps.  We are looking
> for a full backup of the Jenkins and so decided to go with
> PeriodicBackup plugin.  With this plugin, the backup works
> fine.  But, we are trying to restore it in a completely new Jenkins
> setup and not able to restore.  When we pick the backup file that we
> want to restore from, it says restoring, but nothing happens.  Even
> after waiting out for over 30 mins, nothing is restored.  As a work
> around, I stopped the Jenkins service, renamed the 'JENKINS_HOME'
> (/var/lib/jenkins) folder to 'jenkins_backup' and created another
> folder called 'jenkins' in the same location.  In this folder, I
> copied all the files from the backup and then started Jenkins (NOTE:
> I didn't change the .owner file.  I'm using the .owner file of the
> new Jenkins only).  Jenkins came up and I was able to see all the
> configurations from the old one.  Everything was fine.  I let it like
> that overnight and in the morning when I came and tried to access it,
> Jenkins was way too slow and I was not able to save any new
> configuration.  When I try I'm getting
> java.io.ioexception: too many open files jenkins
> error.  I restarted the jenkins and it's taking too long for the UI
> to come up.  Am I doing anything wrong here?  Any suggestions for my
> requirement?
>
> P.S. - I tried SCMSync plugin and ThinBackup.  But, problem with
> these are that they don't do full backup.  But, with ThinBackup both
> backup and restore works fine.
>
> --
> You received this message because you are subscribed to the Google
> Groups "Jenkins Users" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to jenkinsci-use...@googlegroups.com<mailto:
> jenkinsci-use...@googlegroups.com>.
> To view this discussion on the web visit
> <a href="https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com&#39;;return true;"> https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com<<a href="https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com?utm_medium=email&amp;utm_source=footer" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;">https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com?utm_medium=email&utm_source=footer>
> ;.
> For more options, visit <a href="https://groups.google.com/d/optout" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;"> https://groups.google.com/d/optout.
>

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="SdlC-NCqBAAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">jenkinsci-use...@googlegroups.com.
To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/jenkinsci-users/f5625308-1145-4d77-b021-62be8a779592%40googlegroups.com?utm_medium=email&amp;utm_source=footer" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/msgid/jenkinsci-users/f5625308-1145-4d77-b021-62be8a779592%40googlegroups.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/msgid/jenkinsci-users/f5625308-1145-4d77-b021-62be8a779592%40googlegroups.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;"> https://groups.google.com/d/msgid/jenkinsci-users/f5625308-1145-4d77-b021-62be8a779592%40googlegroups.com.
For more options, visit <a href="https://groups.google.com/d/optout" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;">https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/459109a3-0d6e-4780-8381-23c60a991516%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Jenkins Backup and Restore

gotviseryon
In reply to this post by Mark Waite-2
I'm relatively new to all these and I'm just now realizing there are this many ways to use Jenkins (sorry for being dumb).  YES!!! like you said, we want to use a backup mechanism as we don't have any of these in place.  Where can I find documentation or study materials for the stuff you mentioned here?  Would really appreciate if you can point me to some. 

On Tuesday, November 27, 2018 at 12:40:10 PM UTC-5, Mark Waite wrote:
He's saying that if you don't need to backup and restore your job definitions (possibly they are all multibranch Pipeline jobs that are easy to recreate, or they are all defined with the Job DSL plugin so they will be automatically recreated or they are defined in a Docker image) and don't need to backup and restore your credential definitions (same reasons) and don't need to backup and restore other configuration settings (possibly because you're using Configuration as Code plugin to store your configuration in a source control system), then there are backup concepts that use a source control system as your backup system.

In your use case, I suspect you want to use a backup.

Mark Waite

On Tue, Nov 27, 2018 at 7:50 AM <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="79uCc6OrBAAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">gotvi...@...> wrote:
I'm not sure what you are trying to say?

On Monday, November 26, 2018 at 8:03:34 PM UTC-5, rajendraprasad reddy wrote:
Hi ,
I have another solution, just document the list of plugins in old Jenkins server and instead of backup and restore plugins just reinstall plugins on new Jenkins server.

Thanks,
Rajendra

On Fri, 2 Nov, 2018, 02:06 <[hidden email] wrote:
We have a Jenkins instance running and we are trying to setup a backup and restore mechanism in case of any mishaps.  We are looking for a full backup of the Jenkins and so decided to go with PeriodicBackup plugin.  With this plugin, the backup works fine.  But, we are trying to restore it in a completely new Jenkins setup and not able to restore.  When we pick the backup file that we want to restore from, it says restoring, but nothing happens.  Even after waiting out for over 30 mins, nothing is restored.  As a work around, I stopped the Jenkins service, renamed the 'JENKINS_HOME' (/var/lib/jenkins) folder to 'jenkins_backup' and created another folder called 'jenkins' in the same location.  In this folder, I copied all the files from the backup and then started Jenkins (NOTE: I didn't change the .owner file.  I'm using the .owner file of the new Jenkins only).  Jenkins came up and I was able to see all the configurations from the old one.  Everything was fine.  I let it like that overnight and in the morning when I came and tried to access it, Jenkins was way too slow and I was not able to save any new configuration.  When I try I'm getting 
java.io.ioexception: too many open files jenkins
error.  I restarted the jenkins and it's taking too long for the UI to come up.  Am I doing anything wrong here?  Any suggestions for my requirement?

P.S. - I tried SCMSync plugin and ThinBackup.  But, problem with these are that they don't do full backup.  But, with ThinBackup both backup and restore works fine.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.
To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com?utm_medium=email&amp;utm_source=footer" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;">https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com.
For more options, visit <a href="https://groups.google.com/d/optout" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;">https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="79uCc6OrBAAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">jenkinsci-use...@googlegroups.com.
To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/jenkinsci-users/9d3fad8e-7bbe-4472-86dd-57ef3d0da960%40googlegroups.com?utm_medium=email&amp;utm_source=footer" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/msgid/jenkinsci-users/9d3fad8e-7bbe-4472-86dd-57ef3d0da960%40googlegroups.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/msgid/jenkinsci-users/9d3fad8e-7bbe-4472-86dd-57ef3d0da960%40googlegroups.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;">https://groups.google.com/d/msgid/jenkinsci-users/9d3fad8e-7bbe-4472-86dd-57ef3d0da960%40googlegroups.com.
For more options, visit <a href="https://groups.google.com/d/optout" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;">https://groups.google.com/d/optout.


--
Thanks!
Mark Waite

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/ce0af9d3-07d0-4de3-915f-ecff4bf06e47%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Jenkins Backup and Restore

Artur Szostak
In reply to this post by gotviseryon
Do you mean your Jenkins master is hosted on a VM in Azure or the build nodes?
I would not backup the build nodes. I would treat them as volatile and only focus on backing up the master. Unless you have a good reason for backing up the build nodes also.

As for the Jenkins master, I do not know much about Azure, but does it have any kind of copy-on-write snapshot mechanism that you could trigger from within the VM? If yes, then you could call that mechanism from within a periodic backup job within Jenkins, but make sure the job only executes when nothing else is running on Jenkins.

In our case we use Btrfs for the filesystem, so we call something like the following from within a Jenkins job:

  sudo btrfs subvolume snapshot <jenkins_home> <snapshot_path>

To make sure the job only runs when nothing else is running we take advantage of the following plugin:

  https://plugins.jenkins.io/build-blocker-plugin

and then just use '.*' for the regular expression pattern.
After the snapshot is complete, the snapshot itself is backed up to tape with a commercial solution. I believe our IT department does this on the block level.

Cheers

Artur

________________________________________
From: [hidden email] <[hidden email]> on behalf of [hidden email] <[hidden email]>
Sent: 26 November 2018 22:42
To: Jenkins Users
Subject: Re: Jenkins Backup and Restore

Our cloud is Azure.  Do you have any suggestions for this?

On Monday, November 26, 2018 at 4:13:06 PM UTC-5, Artur Szostak wrote:
Just my opinion: but I would forget about the backup plugins in Jenkins for full production backups. They all do a poor job when it comes to a full backup for a production system that also needs high availability. The best that can be done at the moment is to run Jenkins on a file system or data store that is able to perform copy-on-write snapshots. From within Jenkins you will want to setup a simple job that triggers the copy-on-write snapshot when no other job is running. Then use your normal production backup solutions to backup the snapshot in the background.

Cheers

Artur

________________________________________
From: [hidden email] <[hidden email]> on behalf of [hidden email] <[hidden email]>
Sent: 01 November 2018 21:36
To: Jenkins Users
Subject: Jenkins Backup and Restore

We have a Jenkins instance running and we are trying to setup a backup and restore mechanism in case of any mishaps.  We are looking for a full backup of the Jenkins and so decided to go with PeriodicBackup plugin.  With this plugin, the backup works fine.  But, we are trying to restore it in a completely new Jenkins setup and not able to restore.  When we pick the backup file that we want to restore from, it says restoring, but nothing happens.  Even after waiting out for over 30 mins, nothing is restored.  As a work around, I stopped the Jenkins service, renamed the 'JENKINS_HOME' (/var/lib/jenkins) folder to 'jenkins_backup' and created another folder called 'jenkins' in the same location.  In this folder, I copied all the files from the backup and then started Jenkins (NOTE: I didn't change the .owner file.  I'm using the .owner file of the new Jenkins only).  Jenkins came up and I was able to see all the configurations from the old one.  Everything was fine.  I let it like that overnight and in the morning when I came and tried to access it, Jenkins was way too slow and I was not able to save any new configuration.  When I try I'm getting
java.io.ioexception: too many open files jenkins
error.  I restarted the jenkins and it's taking too long for the UI to come up.  Am I doing anything wrong here?  Any suggestions for my requirement?

P.S. - I tried SCMSync plugin and ThinBackup.  But, problem with these are that they don't do full backup.  But, with ThinBackup both backup and restore works fine.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email]<mailto:[hidden email]>.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com<https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com?utm_medium=email&utm_source=footer>.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email]<mailto:[hidden email]>.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/7c46683a-fbfb-4d79-b470-10246e8999d6%40googlegroups.com<https://groups.google.com/d/msgid/jenkinsci-users/7c46683a-fbfb-4d79-b470-10246e8999d6%40googlegroups.com?utm_medium=email&utm_source=footer>.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/c643a72943554a04bff8a5d0064f18d7%40partner.eso.org.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Jenkins Backup and Restore

Artur Szostak
In reply to this post by jxpearce
None of the plugins performs a copy-on-write style full backup of $JENKINS_HOME on the master (I am assuming workspaces are not included or stored somewhere else). Why copy-on-write: because I want close to zero down time in my Jenkins master when performing the backup. I also see no evidence of any of the backup plugins applying a proper quiesce strategy.


________________________________________
From: [hidden email] <[hidden email]> on behalf of Jeff Pearce <[hidden email]>
Sent: 27 November 2018 18:24
To: [hidden email]
Subject: Re: Jenkins Backup and Restore

What issues are you seeing that prevent them from being good for production scale?

From: <[hidden email]> on behalf of "[hidden email]" <[hidden email]>
Reply-To: "[hidden email]" <[hidden email]>
Date: Tuesday, November 27, 2018 at 6:49 AM
To: Jenkins Users <[hidden email]>
Subject: Re: Jenkins Backup and Restore

Tried out a bunch of them.  Unfortunately, none of them are good for a production scale.

On Monday, November 26, 2018 at 5:18:04 PM UTC-5, [hidden email] wrote:
The beauty of Jenkins is that it's NOT built-in, but can be easily added as a plugin (of which there are several).

On Monday, November 26, 2018 at 1:41:19 PM UTC-8, [hidden email] wrote:
I wish for a tool like Jenkins it had a backup and restore option built in.

On Monday, November 26, 2018 at 4:35:04 PM UTC-5, john.mellor wrote:
I have had restore issues with *ALL* of the backup plugins that I have
tried.  The problem is that they are using stupid mechanisms to copy
the filesystem that cannot correctly copy a symlink.  This breaks the
LastSuccessfulBuild and LastUnsuccessfulBuild and LastFailedBuild
symlinks, causing lots of secondary issues.

Doing my own using cpio or rsync in a cronjob only messes up copying
very large files that exceed the archive limits (4GB for cpio for
instance).


On Mon, 2018-11-26 at 21:12 +0000, Artur Szostak wrote:

> Just my opinion: but I would forget about the backup plugins in
> Jenkins for full production backups. They all do a poor job when it
> comes to a full backup for a production system that also needs high
> availability. The best that can be done at the moment is to run
> Jenkins on a file system or data store that is able to perform copy-
> on-write snapshots. From within Jenkins you will want to setup a
> simple job that triggers the copy-on-write snapshot when no other job
> is running. Then use your normal production backup solutions to
> backup the snapshot in the background.
>
> Cheers
>
> Artur
>
> ________________________________________
> From: [hidden email] <
> [hidden email]> on behalf of [hidden email]
> <[hidden email]>
> Sent: 01 November 2018 21:36
> To: Jenkins Users
> Subject: Jenkins Backup and Restore
>
> We have a Jenkins instance running and we are trying to setup a
> backup and restore mechanism in case of any mishaps.  We are looking
> for a full backup of the Jenkins and so decided to go with
> PeriodicBackup plugin.  With this plugin, the backup works
> fine.  But, we are trying to restore it in a completely new Jenkins
> setup and not able to restore.  When we pick the backup file that we
> want to restore from, it says restoring, but nothing happens.  Even
> after waiting out for over 30 mins, nothing is restored.  As a work
> around, I stopped the Jenkins service, renamed the 'JENKINS_HOME'
> (/var/lib/jenkins) folder to 'jenkins_backup' and created another
> folder called 'jenkins' in the same location.  In this folder, I
> copied all the files from the backup and then started Jenkins (NOTE:
> I didn't change the .owner file.  I'm using the .owner file of the
> new Jenkins only).  Jenkins came up and I was able to see all the
> configurations from the old one.  Everything was fine.  I let it like
> that overnight and in the morning when I came and tried to access it,
> Jenkins was way too slow and I was not able to save any new
> configuration.  When I try I'm getting
> java.io.ioexception: too many open files jenkins
> error.  I restarted the jenkins and it's taking too long for the UI
> to come up.  Am I doing anything wrong here?  Any suggestions for my
> requirement?
>
> P.S. - I tried SCMSync plugin and ThinBackup.  But, problem with
> these are that they don't do full backup.  But, with ThinBackup both
> backup and restore works fine.
>
> --
> You received this message because you are subscribed to the Google
> Groups "Jenkins Users" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to [hidden email]<mailto:
> [hidden email]>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com<https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com?utm_medium=email&utm_source=footer>
> ;.
> For more options, visit https://groups.google.com/d/optout.
>
--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email]<mailto:[hidden email]>.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/f5625308-1145-4d77-b021-62be8a779592%40googlegroups.com<https://groups.google.com/d/msgid/jenkinsci-users/f5625308-1145-4d77-b021-62be8a779592%40googlegroups.com?utm_medium=email&utm_source=footer>.
For more options, visit https://groups.google.com/d/optout.


--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email]<mailto:[hidden email]>.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/588AA5F3-DCE6-4516-B0C7-7E2C161E3D5C%40godaddy.com<https://groups.google.com/d/msgid/jenkinsci-users/588AA5F3-DCE6-4516-B0C7-7E2C161E3D5C%40godaddy.com?utm_medium=email&utm_source=footer>.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/b390f03cc61741098a8da940c4badeaf%40partner.eso.org.
For more options, visit https://groups.google.com/d/optout.