Is there a Jenkins plugin that can create and terminate Amazon EC2 instances as required by the build queue?

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Is there a Jenkins plugin that can create and terminate Amazon EC2 instances as required by the build queue?

Amedee Van Gasse
I have a job that makes use of Tesseract for OCR.
Tesseract runs *a lot* faster on GPU-optimized computers.
So I was thinking of running this job on an AWS EC2 instance of type P or G, which have NVIDIA Tesla GPUs.
However, these instance types are quite expensive, and the job runs only about 10 times per day, for an estimated duration of ~10 minutes.
So it would be very costly to have an AWS EC2 instance online 24/7.

Is there a way in Jenkins, maybe by means of some plugin, that I can for example put in my Jenkinsfile:

agent { label 'tesseract' }

and that when a job with label 'tesseracr' enters the queue, Jenkins connects to AWS, starts an instance based off an already existing image, runs the job on that image, and then terminates the instance?


--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/25a9ac47-fdb7-4f1b-a546-73b014d9cb19o%40googlegroups.com.
Reply | Threaded
Open this post in threaded view
|

Re: Is there a Jenkins plugin that can create and terminate Amazon EC2 instances as required by the build queue?

alok kumar
Hi,
I guess what you are looking for is this https://plugins.jenkins.io/ec2/

This should be able to help you achieve your requirement.

Thanks


On Tue., Jul. 14, 2020, 04:22 Amedee Van Gasse, <[hidden email]> wrote:
I have a job that makes use of Tesseract for OCR.
Tesseract runs *a lot* faster on GPU-optimized computers.
So I was thinking of running this job on an AWS EC2 instance of type P or G, which have NVIDIA Tesla GPUs.
However, these instance types are quite expensive, and the job runs only about 10 times per day, for an estimated duration of ~10 minutes.
So it would be very costly to have an AWS EC2 instance online 24/7.

Is there a way in Jenkins, maybe by means of some plugin, that I can for example put in my Jenkinsfile:

agent { label 'tesseract' }

and that when a job with label 'tesseracr' enters the queue, Jenkins connects to AWS, starts an instance based off an already existing image, runs the job on that image, and then terminates the instance?


--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/25a9ac47-fdb7-4f1b-a546-73b014d9cb19o%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CAOqqq4QT_T62crWtpWHozfH1QbvgynF%3DLa3o4oHzFvdOKj%2BSaQ%40mail.gmail.com.
Reply | Threaded
Open this post in threaded view
|

Re: Is there a Jenkins plugin that can create and terminate Amazon EC2 instances as required by the build queue?

Ramanathan Muthaiah
In reply to this post by Amedee Van Gasse
On Tuesday, July 14, 2020 at 1:51:47 PM UTC+5:30, Amedee Van Gasse wrote:
Is there a way in Jenkins, maybe by means of some plugin, that I can for example put in my Jenkinsfile:

agent { label 'tesseract' }

and that when a job with label 'tesseracr' enters the queue, Jenkins connects to AWS, starts an instance based off an already existing image, runs the job on that image, and then terminates the instance?

Adding to what has been confirmed by Alok Kumar, with the plugin installed, you can set fixed uptime for the instances to be alive. Post that, they will be destroyed.

/Ram

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/4c7cd662-b7cb-4f5c-b426-eceb237b45d3o%40googlegroups.com.