Testing Jenkins Plugins

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Testing Jenkins Plugins

Valerie Fernandes
Hi Guys,

I need your advise to understand what is the best means to test Jenkins plugins. 
We have some plugins that were developed in house. To test the functionality we have selenium tests per plugin.
Problem is that, these tests are time consuming, and often fail due to timeout issues or element not found and could succeed on a rerun.
Additionally, changes in UI(could be due to Jenkins upgrade 1.x --> 2.x) means we need to make changes in the tests as well. :( 

I see most Jenkins plugins have JUnit's. Can you suggest?

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/c9c9e540-27c2-451d-9fc9-2c0562bef9db%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Testing Jenkins Plugins

Mark Waite-2
There is a great thread earlier in the list which offers insights into the different facilities available to test a plugin, the concepts involved, and the related compromises.


Mark Waite

On Tue, Oct 24, 2017 at 7:00 AM Valerie Fernandes <[hidden email]> wrote:
Hi Guys,

I need your advise to understand what is the best means to test Jenkins plugins. 
We have some plugins that were developed in house. To test the functionality we have selenium tests per plugin.
Problem is that, these tests are time consuming, and often fail due to timeout issues or element not found and could succeed on a rerun.
Additionally, changes in UI(could be due to Jenkins upgrade 1.x --> 2.x) means we need to make changes in the tests as well. :( 

I see most Jenkins plugins have JUnit's. Can you suggest?

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/c9c9e540-27c2-451d-9fc9-2c0562bef9db%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAO49JtH9Pp1FLc0c9Grp1oeXrZWb17jDiffVDmVTepUX21v-5g%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Testing Jenkins Plugins

Valerie Fernandes
Thanks Mark,

So what I understand is,
  • Primarily use JUnits based on JTH in order to run most of the tests for functionality. 
  • Use ATH minimally for testing the integration of plugins with each other
Please correct me if I have understood incorrectly.

On Tuesday, 24 October 2017 18:44:00 UTC+5:30, Mark Waite wrote:
There is a great thread earlier in the list which offers insights into the different facilities available to test a plugin, the concepts involved, and the related compromises.

Refer to <a href="https://groups.google.com/d/msg/jenkinsci-dev/R8w7uhh-xbg/Z3O5ew1NBwAJ" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/msg/jenkinsci-dev/R8w7uhh-xbg/Z3O5ew1NBwAJ&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/msg/jenkinsci-dev/R8w7uhh-xbg/Z3O5ew1NBwAJ&#39;;return true;">https://groups.google.com/d/msg/jenkinsci-dev/R8w7uhh-xbg/Z3O5ew1NBwAJ

Mark Waite

On Tue, Oct 24, 2017 at 7:00 AM Valerie Fernandes <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="Y9hUeeRlBwAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">valer...@...> wrote:
Hi Guys,

I need your advise to understand what is the best means to test Jenkins plugins. 
We have some plugins that were developed in house. To test the functionality we have selenium tests per plugin.
Problem is that, these tests are time consuming, and often fail due to timeout issues or element not found and could succeed on a rerun.
Additionally, changes in UI(could be due to Jenkins upgrade 1.x --> 2.x) means we need to make changes in the tests as well. :( 

I see most Jenkins plugins have JUnit's. Can you suggest?

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="Y9hUeeRlBwAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/jenkinsci-dev/c9c9e540-27c2-451d-9fc9-2c0562bef9db%40googlegroups.com?utm_medium=email&amp;utm_source=footer" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/msgid/jenkinsci-dev/c9c9e540-27c2-451d-9fc9-2c0562bef9db%40googlegroups.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/msgid/jenkinsci-dev/c9c9e540-27c2-451d-9fc9-2c0562bef9db%40googlegroups.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;">https://groups.google.com/d/msgid/jenkinsci-dev/c9c9e540-27c2-451d-9fc9-2c0562bef9db%40googlegroups.com.
For more options, visit <a href="https://groups.google.com/d/optout" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;">https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/421affea-5557-43a1-8da0-41d6cfdb2c81%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Testing Jenkins Plugins

Mark Waite-2
You've described the strategy that I prefer.

Mark Waite

On Wed, Oct 25, 2017 at 3:04 AM Valerie Fernandes <[hidden email]> wrote:
Thanks Mark,

So what I understand is,
  • Primarily use JUnits based on JTH in order to run most of the tests for functionality. 
  • Use ATH minimally for testing the integration of plugins with each other
Please correct me if I have understood incorrectly.

On Tuesday, 24 October 2017 18:44:00 UTC+5:30, Mark Waite wrote:
There is a great thread earlier in the list which offers insights into the different facilities available to test a plugin, the concepts involved, and the related compromises.


Mark Waite

On Tue, Oct 24, 2017 at 7:00 AM Valerie Fernandes <[hidden email]> wrote:
Hi Guys,

I need your advise to understand what is the best means to test Jenkins plugins. 
We have some plugins that were developed in house. To test the functionality we have selenium tests per plugin.
Problem is that, these tests are time consuming, and often fail due to timeout issues or element not found and could succeed on a rerun.
Additionally, changes in UI(could be due to Jenkins upgrade 1.x --> 2.x) means we need to make changes in the tests as well. :( 

I see most Jenkins plugins have JUnit's. Can you suggest?

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/421affea-5557-43a1-8da0-41d6cfdb2c81%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAO49JtFqC61vAifk4zxBb0ChjaqFY7-oBLPaZbtQ8ju572NTKA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Testing Jenkins Plugins

Jesse Glick-4
In reply to this post by Valerie Fernandes
On Wed, Oct 25, 2017 at 3:15 AM, Valerie Fernandes
<[hidden email]> wrote:
> Use ATH minimally for testing the integration of plugins with each other

I actually prefer to do most integration testing using `JenkinsRule`
as well—you simply add a `test`-scoped dependency on another plugin
(if not already in your dependency tree anyway), and you are good to
go.

ATH is good for tests that specifically cover the UI, or that demand
an unusually high fidelity to production behaviors in certain areas
such as class loading.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr1ZFmR-iDLzF7RdXJ_Z7ZMYv1Ao7q_rp%2BaQEF-k2buCew%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Testing Jenkins Plugins

Mark Waite-2


On Wed, Oct 25, 2017 at 10:31 AM Jesse Glick <[hidden email]> wrote:
On Wed, Oct 25, 2017 at 3:15 AM, Valerie Fernandes
<[hidden email]> wrote:
> Use ATH minimally for testing the integration of plugins with each other

I actually prefer to do most integration testing using `JenkinsRule`
as well—you simply add a `test`-scoped dependency on another plugin
(if not already in your dependency tree anyway), and you are good to
go.

ATH is good for tests that specifically cover the UI, or that demand
an unusually high fidelity to production behaviors in certain areas
such as class loading.


To further support Jesse's comment, I use JTH based tests with JenkinsRule for git plugin and git client plugin tests.  I don't use any ATH based tests for git plugin or git client plugin.  There are others who have written ATH tests of git plugin and git client plugin.

Mark Waite
 
--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr1ZFmR-iDLzF7RdXJ_Z7ZMYv1Ao7q_rp%2BaQEF-k2buCew%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAO49JtFtfV7YTDmjqParYR7BoqNqTRg9b_ujYSMBdwrwfsss4w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Testing Jenkins Plugins

Jesse Glick-4
In reply to this post by Jesse Glick-4
On Wed, Oct 25, 2017 at 12:31 PM, Jesse Glick <[hidden email]> wrote:
> I actually prefer to do most integration testing using `JenkinsRule`

I suppose I should clarify that this refers to testing a specific
feature or fix with awareness of its point of implementation. There is
also a use case for ATH in terms of smoke-testing an overall user
workflow from end to end looking for otherwise unforeseeable
regressions or poor interactions among numerous components, but that
is probably not what you were thinking of when you asked about testing
a particular plugin.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr1-q2nGAVWsogR%3DKVvyWKg4vV540W7La1s2NwUAwEcXcQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.