How to get workspace folder content of slave nodes?

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

How to get workspace folder content of slave nodes?

Xiaocheng Wang
Hello Jenkins team,

We have deployed 1 Jenkins master node + 3 slave node. Jobs are building on different slaves, and generates some artifacts in the workspace. We wonder info there's a way (or API) to get the workspace folder content of EACH running build in realtime?

We tried to get this info by URL "http://<jenkins_host>:<jenkins_port>/job/<job_name>/ws", however it only displays workspace content of slave node which runs the latest build. (If 2 jobs are running in parallel on different slave, we are unable to get the first one's workspace content by that URL.)


--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/25e7fd40-257e-47ea-8fb2-f9c6c3b24f84%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: How to get workspace folder content of slave nodes?

Jesse Glick-4
On Mon, Oct 8, 2018 at 7:55 AM 'Xiaocheng Wang' via Jenkins Developers
<[hidden email]> wrote:
> We wonder info there's a way (or API) to get the workspace folder content of EACH running build in realtime?

Using the Java APIs, easily. Using HTTP APIs…depends. It would be
helpful if you clarified why you are posting to the dev list. Are you
writing a plugin, or something else?

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr2wKzC181RP9AE%3D%3DnwfmW%3D1Xa%2BJ0Z4uQoo%3D3kBW5JbNVw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: How to get workspace folder content of slave nodes?

Xiaocheng Wang
Many thanks. The background is we have an Java application that calls Jenkins REST API to trigger builds. Due to project needs, we want that application to get the slave workspace folder content when build is running. 

So I think either calling a REST/HTTP API or developing a plugin ourselves may be the solution. But using REST/HTTP API (if there is) will be easy.

I originally posted in jenkinsci-users but haven't got an answer.
https://groups.google.com/forum/#!starred/jenkinsci-users/vszuG8lyOlI

On Monday, October 8, 2018 at 10:21:06 PM UTC+8, Jesse Glick wrote:
On Mon, Oct 8, 2018 at 7:55 AM 'Xiaocheng Wang' via Jenkins Developers
<<a href="javascript:" target="_blank" gdf-obfuscated-mailto="zELzoRbRAgAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">jenkin...@googlegroups.com> wrote:
> We wonder info there's a way (or API) to get the workspace folder content of EACH running build in realtime?

Using the Java APIs, easily. Using HTTP APIs…depends. It would be
helpful if you clarified why you are posting to the dev list. Are you
writing a plugin, or something else?

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/27c3eaeb-5176-49c2-9b1d-8f497aad194d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: How to get workspace folder content of slave nodes?

Jesse Glick-4
On Tue, Oct 9, 2018 at 12:53 AM 'Xiaocheng Wang' via Jenkins
Developers <[hidden email]> wrote:
> we have an Java application that calls Jenkins REST API to trigger builds. Due to project needs, we want that application to get the slave workspace folder content when build is running.

This sounds inherently unreliable. Treat the workspace as private. If
you want some files from a build, archive them.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr0Sqy_Lpor7Vp7SFipH5ePfNV6eihwgXx0GrW3YPOYomA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: How to get workspace folder content of slave nodes?

Xiaocheng Wang
Not sure what does "treat the workspace as private" exactly mean. If it's private, why can URL "http://<jenkins_host>:<jenkins_port>/job/<job_name>/ws" display those info?

And Yes, we archived the files after the build completes. However this is not enough -- users want to get those files when build is running so that he can know the test progress and responses in real time. Any suggestions to implement this?



On Tuesday, October 9, 2018 at 9:18:55 PM UTC+8, Jesse Glick wrote:
On Tue, Oct 9, 2018 at 12:53 AM 'Xiaocheng Wang' via Jenkins
Developers <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="5yNOjEYcAwAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">jenkin...@googlegroups.com> wrote:
> we have an Java application that calls Jenkins REST API to trigger builds. Due to project needs, we want that application to get the slave workspace folder content when build is running.

This sounds inherently unreliable. Treat the workspace as private. If
you want some files from a build, archive them.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/b293e9ac-03b1-4e9e-895f-e86a185e33bb%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: How to get workspace folder content of slave nodes?

Jesse Glick-4
On Tue, Oct 9, 2018 at 10:00 PM 'Xiaocheng Wang' via Jenkins
Developers <[hidden email]> wrote:
> Not sure what does "treat the workspace as private" exactly mean. If it's private, why can URL "http://<jenkins_host>:<jenkins_port>/job/<job_name>/ws" display those info?

The workspace browser is intended only for interactive
troubleshooting. By design, the workspace is only guaranteed
accessible during the build (in the case of Pipeline, during a `node`
block), as subsequent builds may overwrite these files, and “cloudy”
agents will typically throw away the whole filesystem as soon as the
build is done.

> users want to get those files when build is running so that he can know the test progress and responses in real time.

In the case of JUnit-formatted test results, this is handled via

https://plugins.jenkins.io/junit-realtime-test-reporter

This is done as a plugin, so it uses Java APIs to access these files.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr2PKqYSU88Spqf-BVpG9Q8ConRvNHpRGXSsQ5qnuTqBOA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: How to get workspace folder content of slave nodes?

Xiaocheng Wang
Yes, I fully understand the workspace is only guaranteed accessible when the build is running. That's why we archived the files after the build completes.

However the plugin you sent does not meet our requirement... First, our test is not junit test, but jmeter test. And, during the build, we do not want the test results, but want to get the artifacts generated by jmeter (e.g. csv file, typically stored in the workspace folder of slave node). Anyway suggestions to handle this?

On Wednesday, October 10, 2018 at 10:10:12 PM UTC+8, Jesse Glick wrote:
On Tue, Oct 9, 2018 at 10:00 PM 'Xiaocheng Wang' via Jenkins
Developers <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="lGELY_AXAwAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">jenkin...@googlegroups.com> wrote:
> Not sure what does "treat the workspace as private" exactly mean. If it's private, why can URL "http://<jenkins_host>:<jenkins_port>/job/<job_name>/ws" display those info?

The workspace browser is intended only for interactive
troubleshooting. By design, the workspace is only guaranteed
accessible during the build (in the case of Pipeline, during a `node`
block), as subsequent builds may overwrite these files, and “cloudy”
agents will typically throw away the whole filesystem as soon as the
build is done.

> users want to get those files when build is running so that he can know the test progress and responses in real time.

In the case of JUnit-formatted test results, this is handled via

<a href="https://plugins.jenkins.io/junit-realtime-test-reporter" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fplugins.jenkins.io%2Fjunit-realtime-test-reporter\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNE4Z6h4YzUJQu1NLgiVbsp-mytWYQ&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fplugins.jenkins.io%2Fjunit-realtime-test-reporter\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNE4Z6h4YzUJQu1NLgiVbsp-mytWYQ&#39;;return true;">https://plugins.jenkins.io/junit-realtime-test-reporter

This is done as a plugin, so it uses Java APIs to access these files.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/ec469ce3-5d3e-4bee-b63c-f67da8e93b82%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: How to get workspace folder content of slave nodes?

slide
This is really a question for the users list, not the developers list.

On Wed, Oct 10, 2018, 20:11 'Xiaocheng Wang' via Jenkins Developers <[hidden email]> wrote:
Yes, I fully understand the workspace is only guaranteed accessible when the build is running. That's why we archived the files after the build completes.

However the plugin you sent does not meet our requirement... First, our test is not junit test, but jmeter test. And, during the build, we do not want the test results, but want to get the artifacts generated by jmeter (e.g. csv file, typically stored in the workspace folder of slave node). Anyway suggestions to handle this?

On Wednesday, October 10, 2018 at 10:10:12 PM UTC+8, Jesse Glick wrote:
On Tue, Oct 9, 2018 at 10:00 PM 'Xiaocheng Wang' via Jenkins
Developers <[hidden email]> wrote:
> Not sure what does "treat the workspace as private" exactly mean. If it's private, why can URL "http://<jenkins_host>:<jenkins_port>/job/<job_name>/ws" display those info?

The workspace browser is intended only for interactive
troubleshooting. By design, the workspace is only guaranteed
accessible during the build (in the case of Pipeline, during a `node`
block), as subsequent builds may overwrite these files, and “cloudy”
agents will typically throw away the whole filesystem as soon as the
build is done.

> users want to get those files when build is running so that he can know the test progress and responses in real time.

In the case of JUnit-formatted test results, this is handled via

https://plugins.jenkins.io/junit-realtime-test-reporter

This is done as a plugin, so it uses Java APIs to access these files.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/ec469ce3-5d3e-4bee-b63c-f67da8e93b82%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAPiUgVeeBevm3PyE2Zn-Z%2BTgGeAXRHiN4ExMc96Onfu6k8byCQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: How to get workspace folder content of slave nodes?

Jesse Glick-4
In reply to this post by Xiaocheng Wang
On Wed, Oct 10, 2018 at 11:11 PM 'Xiaocheng Wang' via Jenkins
Developers <[hidden email]> wrote:
> our test is not junit test, but jmeter test. And, during the build, we do not want the test results, but want to get the artifacts generated by jmeter (e.g. csv file, typically stored in the workspace folder of slave node). Anyway suggestions to handle this?

Either modify your project to periodically upload those CSV files
somewhere, and have a standalone dashboard to display them; or write a
plugin to collect them automatically and publish them on the build
page using an `Action`.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr1taLDyFeB0KnaNmsAxtsGZmw0-FhLT4Dau3ZES91qZKA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.