Quantcast

How to copy artifacts back to workspaces of upstream job?

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

How to copy artifacts back to workspaces of upstream job?

intelchen
Hi all,
       There are two stage jobs for one java project.
One is to do compilation job and create a runtime jar file after polling codes from SCM.
Second one is to do findbug analysis job. It copy artifact(runtime jar file) from upstream using Copy Artifact Plugin. And there would be a findbug_result.xml generated after this job.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: How to copy artifacts back to workspaces of upstream job?

intelchen
Sorry for a uncompleted mail.
And I would like to copy the findbug_result.xml back to the workspaces of the first job.
I think there would be a way to do that is write some shell scripts and copy this result back to the former job.
But is there any other solution or existed plug-ins for this requirement?
May I have your ideas? Thanks!

On Friday, March 16, 2012 3:23:30 PM UTC+8, intelchen wrote:
Hi all,
       There are two stage jobs for one java project.
One is to do compilation job and create a runtime jar file after polling codes from SCM.
Second one is to do findbug analysis job. It copy artifact(runtime jar file) from upstream using Copy Artifact Plugin. And there would be a findbug_result.xml generated after this job.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: How to copy artifacts back to workspaces of upstream job?

Sami Tikka
The most straightforward way is to run the findbug analysis in the same job. Why does it need to be run in its own job?

Jenkins jobs are not as flexible as subroutines in programming languages. If you start using them as such, you will shoot yourself in the foot. 

-- Sami

intelchen <[hidden email]> kirjoitti 16.3.2012 kello 9.28:

Sorry for a uncompleted mail.
And I would like to copy the findbug_result.xml back to the workspaces of the first job.
I think there would be a way to do that is write some shell scripts and copy this result back to the former job.
But is there any other solution or existed plug-ins for this requirement?
May I have your ideas? Thanks!

On Friday, March 16, 2012 3:23:30 PM UTC+8, intelchen wrote:
Hi all,
       There are two stage jobs for one java project.
One is to do compilation job and create a runtime jar file after polling codes from SCM.
Second one is to do findbug analysis job. It copy artifact(runtime jar file) from upstream using Copy Artifact Plugin. And there would be a findbug_result.xml generated after this job.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: How to copy artifacts back to workspaces of upstream job?

intelchen
Hi,
     It would take so much time to get results of these static code analysis tools.
For example, it takes 12minutes to get findbugs result of our one component. And we have more than 20 components in our products.
And aslo we would like to use checkstyles,pmd besides findbugs.
So I break apart them into two stage jobs. The fist stage is about compliation. The second stage is about these tools.
And these tools could run in parallel.Then we could get the results more quickly,  and send the reports to developers in short time.
   Is it a bad or good practice in Jenkins?

Brs,
Bill

On Friday, March 16, 2012 8:59:13 PM UTC+8, sti wrote:
The most straightforward way is to run the findbug analysis in the same job. Why does it need to be run in its own job?

Jenkins jobs are not as flexible as subroutines in programming languages. If you start using them as such, you will shoot yourself in the foot. 

-- Sami

intelchen <[hidden email]> kirjoitti 16.3.2012 kello 9.28:

Sorry for a uncompleted mail.
And I would like to copy the findbug_result.xml back to the workspaces of the first job.
I think there would be a way to do that is write some shell scripts and copy this result back to the former job.
But is there any other solution or existed plug-ins for this requirement?
May I have your ideas? Thanks!

On Friday, March 16, 2012 3:23:30 PM UTC+8, intelchen wrote:
Hi all,
       There are two stage jobs for one java project.
One is to do compilation job and create a runtime jar file after polling codes from SCM.
Second one is to do findbug analysis job. It copy artifact(runtime jar file) from upstream using Copy Artifact Plugin. And there would be a findbug_result.xml generated after this job.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: How to copy artifacts back to workspaces of upstream job?

Sami Tikka
I typically try to run as much in one job as possible to minimize the number of jobs. I use these advanced unix operating systems that allow me to run several things at once like this:

Compile &
Analyze &
wait

If you must split it into multiple jobs, it is possible and even desirable if it allows you to get faster feedback. The only thing is, you cannot send the results upstream. If you have to have a single job where everything is available, both build artifacts and test results, you need to make a new job downstream that collects them both. 

There used to be this option to aggregate test results, which kind of does what you want but I never got it to work. Maybe that option is not compatible with multiconfiguration jobs which in use a lot. 

-- Sami

intelchen <[hidden email]> kirjoitti 16.3.2012 kello 18.03:

Hi,
     It would take so much time to get results of these static code analysis tools.
For example, it takes 12minutes to get findbugs result of our one component. And we have more than 20 components in our products.
And aslo we would like to use checkstyles,pmd besides findbugs.
So I break apart them into two stage jobs. The fist stage is about compliation. The second stage is about these tools.
And these tools could run in parallel.Then we could get the results more quickly,  and send the reports to developers in short time.
   Is it a bad or good practice in Jenkins?

Brs,
Bill

On Friday, March 16, 2012 8:59:13 PM UTC+8, sti wrote:
The most straightforward way is to run the findbug analysis in the same job. Why does it need to be run in its own job?

Jenkins jobs are not as flexible as subroutines in programming languages. If you start using them as such, you will shoot yourself in the foot. 

-- Sami

intelchen <[hidden email]> kirjoitti 16.3.2012 kello 9.28:

Sorry for a uncompleted mail.
And I would like to copy the findbug_result.xml back to the workspaces of the first job.
I think there would be a way to do that is write some shell scripts and copy this result back to the former job.
But is there any other solution or existed plug-ins for this requirement?
May I have your ideas? Thanks!

On Friday, March 16, 2012 3:23:30 PM UTC+8, intelchen wrote:
Hi all,
       There are two stage jobs for one java project.
One is to do compilation job and create a runtime jar file after polling codes from SCM.
Second one is to do findbug analysis job. It copy artifact(runtime jar file) from upstream using Copy Artifact Plugin. And there would be a findbug_result.xml generated after this job.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: How to copy artifacts back to workspaces of upstream job?

danny-orion
I suspect the answer here may be to run the child jobs in a build step, while the parent job waits, and then gravs the downstream jobs artifacts via url. The cli can be used for synchronous job runs, and with a newer jenkins you may even be able to use the SSH system to do it (saves grabbing a copy of that cli jar).

You can actually combine those synchronous runs with wgets in a shell script using the same facilities sti has mentioned:

(java -jar jenkins-cli.jar $JENKINS_URL build compile_job && wget <artifact name>) &
(java -jar jenkins-cli.jar $JENKINS_URL build analyze_job && wget <artifact name>) &
wait

I've generalized, and you'll want to look at the cli commands on your server (read https://wiki.jenkins-ci.org/display/JENKINS/Jenkins+CLI) for info on it. I also use a for pid in `jobs` style thing if I've many unix jobs on a shell I want to wait for - again, you'll want to familiarise yourself with the job handling syntax of your shell or build language for such things.

On Friday, 16 March 2012 17:54:27 UTC, sti wrote:
I typically try to run as much in one job as possible to minimize the number of jobs. I use these advanced unix operating systems that allow me to run several things at once like this:

Compile &
Analyze &
wait

If you must split it into multiple jobs, it is possible and even desirable if it allows you to get faster feedback. The only thing is, you cannot send the results upstream. If you have to have a single job where everything is available, both build artifacts and test results, you need to make a new job downstream that collects them both. 

There used to be this option to aggregate test results, which kind of does what you want but I never got it to work. Maybe that option is not compatible with multiconfiguration jobs which in use a lot. 

-- Sami

intelchen <[hidden email]> kirjoitti 16.3.2012 kello 18.03:

Hi,
     It would take so much time to get results of these static code analysis tools.
For example, it takes 12minutes to get findbugs result of our one component. And we have more than 20 components in our products.
And aslo we would like to use checkstyles,pmd besides findbugs.
So I break apart them into two stage jobs. The fist stage is about compliation. The second stage is about these tools.
And these tools could run in parallel.Then we could get the results more quickly,  and send the reports to developers in short time.
   Is it a bad or good practice in Jenkins?

Brs,
Bill

On Friday, March 16, 2012 8:59:13 PM UTC+8, sti wrote:
The most straightforward way is to run the findbug analysis in the same job. Why does it need to be run in its own job?

Jenkins jobs are not as flexible as subroutines in programming languages. If you start using them as such, you will shoot yourself in the foot. 

-- Sami

intelchen <[hidden email]> kirjoitti 16.3.2012 kello 9.28:

Sorry for a uncompleted mail.
And I would like to copy the findbug_result.xml back to the workspaces of the first job.
I think there would be a way to do that is write some shell scripts and copy this result back to the former job.
But is there any other solution or existed plug-ins for this requirement?
May I have your ideas? Thanks!

On Friday, March 16, 2012 3:23:30 PM UTC+8, intelchen wrote:
Hi all,
       There are two stage jobs for one java project.
One is to do compilation job and create a runtime jar file after polling codes from SCM.
Second one is to do findbug analysis job. It copy artifact(runtime jar file) from upstream using Copy Artifact Plugin. And there would be a findbug_result.xml generated after this job.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: How to copy artifacts back to workspaces of upstream job?

intelchen
In reply to this post by Sami Tikka

    If I run several things in background, Jenkins may not capture the output of these commands.
Do you have experiences  about using this  https://wiki.jenkins-ci.org/display/JENKINS/Spawning+processes+from+build ?
   It is easy for me to put everything available in a single job. Then it would be a very large job. It would take a long time to get the results.:( That is my real problem here.
   
On Saturday, March 17, 2012 1:54:27 AM UTC+8, sti wrote:
I typically try to run as much in one job as possible to minimize the number of jobs. I use these advanced unix operating systems that allow me to run several things at once like this:

Compile &
Analyze &
wait

If you must split it into multiple jobs, it is possible and even desirable if it allows you to get faster feedback. The only thing is, you cannot send the results upstream. If you have to have a single job where everything is available, both build artifacts and test results, you need to make a new job downstream that collects them both. 

There used to be this option to aggregate test results, which kind of does what you want but I never got it to work. Maybe that option is not compatible with multiconfiguration jobs which in use a lot. 

-- Sami

intelchen <[hidden email]> kirjoitti 16.3.2012 kello 18.03:

Hi,
     It would take so much time to get results of these static code analysis tools.
For example, it takes 12minutes to get findbugs result of our one component. And we have more than 20 components in our products.
And aslo we would like to use checkstyles,pmd besides findbugs.
So I break apart them into two stage jobs. The fist stage is about compliation. The second stage is about these tools.
And these tools could run in parallel.Then we could get the results more quickly,  and send the reports to developers in short time.
   Is it a bad or good practice in Jenkins?

Brs,
Bill

On Friday, March 16, 2012 8:59:13 PM UTC+8, sti wrote:
The most straightforward way is to run the findbug analysis in the same job. Why does it need to be run in its own job?

Jenkins jobs are not as flexible as subroutines in programming languages. If you start using them as such, you will shoot yourself in the foot. 

-- Sami

intelchen <[hidden email]> kirjoitti 16.3.2012 kello 9.28:

Sorry for a uncompleted mail.
And I would like to copy the findbug_result.xml back to the workspaces of the first job.
I think there would be a way to do that is write some shell scripts and copy this result back to the former job.
But is there any other solution or existed plug-ins for this requirement?
May I have your ideas? Thanks!

On Friday, March 16, 2012 3:23:30 PM UTC+8, intelchen wrote:
Hi all,
       There are two stage jobs for one java project.
One is to do compilation job and create a runtime jar file after polling codes from SCM.
Second one is to do findbug analysis job. It copy artifact(runtime jar file) from upstream using Copy Artifact Plugin. And there would be a findbug_result.xml generated after this job.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: How to copy artifacts back to workspaces of upstream job?

intelchen
In reply to this post by danny-orion
Thank Danny,
I think my main problem here is about how to organize so many jobs for our products. 
Take example, we have ten components, and one compilation job and several analyze jobs for each component.
So it would be more than 40 jobs for a single products( 10components*1compliation+ 10component *3analyze). And we have also 10 branches for this product. It would be 10branches*40(jobs/product)=400jobs..
And also we could not find all analyze reports in a single job/workspace. We have to observe them in every specify analyze job.


Brs,
Bill

On Monday, March 19, 2012 4:39:56 AM UTC+8, Danny Staple wrote:
I suspect the answer here may be to run the child jobs in a build step, while the parent job waits, and then gravs the downstream jobs artifacts via url. The cli can be used for synchronous job runs, and with a newer jenkins you may even be able to use the SSH system to do it (saves grabbing a copy of that cli jar).

You can actually combine those synchronous runs with wgets in a shell script using the same facilities sti has mentioned:

(java -jar jenkins-cli.jar $JENKINS_URL build compile_job && wget <artifact name>) &
(java -jar jenkins-cli.jar $JENKINS_URL build analyze_job && wget <artifact name>) &
wait

I've generalized, and you'll want to look at the cli commands on your server (read https://wiki.jenkins-ci.org/display/JENKINS/Jenkins+CLI) for info on it. I also use a for pid in `jobs` style thing if I've many unix jobs on a shell I want to wait for - again, you'll want to familiarise yourself with the job handling syntax of your shell or build language for such things.

On Friday, 16 March 2012 17:54:27 UTC, sti wrote:
I typically try to run as much in one job as possible to minimize the number of jobs. I use these advanced unix operating systems that allow me to run several things at once like this:

Compile &
Analyze &
wait

If you must split it into multiple jobs, it is possible and even desirable if it allows you to get faster feedback. The only thing is, you cannot send the results upstream. If you have to have a single job where everything is available, both build artifacts and test results, you need to make a new job downstream that collects them both. 

There used to be this option to aggregate test results, which kind of does what you want but I never got it to work. Maybe that option is not compatible with multiconfiguration jobs which in use a lot. 

-- Sami

intelchen <[hidden email]> kirjoitti 16.3.2012 kello 18.03:

Hi,
     It would take so much time to get results of these static code analysis tools.
For example, it takes 12minutes to get findbugs result of our one component. And we have more than 20 components in our products.
And aslo we would like to use checkstyles,pmd besides findbugs.
So I break apart them into two stage jobs. The fist stage is about compliation. The second stage is about these tools.
And these tools could run in parallel.Then we could get the results more quickly,  and send the reports to developers in short time.
   Is it a bad or good practice in Jenkins?

Brs,
Bill

On Friday, March 16, 2012 8:59:13 PM UTC+8, sti wrote:
The most straightforward way is to run the findbug analysis in the same job. Why does it need to be run in its own job?

Jenkins jobs are not as flexible as subroutines in programming languages. If you start using them as such, you will shoot yourself in the foot. 

-- Sami

intelchen <[hidden email]> kirjoitti 16.3.2012 kello 9.28:

Sorry for a uncompleted mail.
And I would like to copy the findbug_result.xml back to the workspaces of the first job.
I think there would be a way to do that is write some shell scripts and copy this result back to the former job.
But is there any other solution or existed plug-ins for this requirement?
May I have your ideas? Thanks!

On Friday, March 16, 2012 3:23:30 PM UTC+8, intelchen wrote:
Hi all,
       There are two stage jobs for one java project.
One is to do compilation job and create a runtime jar file after polling codes from SCM.
Second one is to do findbug analysis job. It copy artifact(runtime jar file) from upstream using Copy Artifact Plugin. And there would be a findbug_result.xml generated after this job.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: How to copy artifacts back to workspaces of upstream job?

danny staple

It may take a bit more complication, but you could probably use parameterized builds with allow concurrent runs to keep the number of jobs down. We do this. You will want a few coping strategies.

When you kick off sub builds make sure you get the build number as you'll need it to find your artifacts. Use the build description to hold key info about the build so you can spot the right ones in a list of builds.

typed on my phone expect typos...

On 19 Mar 2012 08:59, "intelchen" <[hidden email]> wrote:
Thank Danny,
I think my main problem here is about how to organize so many jobs for our products. 
Take example, we have ten components, and one compilation job and several analyze jobs for each component.
So it would be more than 40 jobs for a single products( 10components*1compliation+ 10component *3analyze). And we have also 10 branches for this product. It would be 10branches*40(jobs/product)=400jobs..
And also we could not find all analyze reports in a single job/workspace. We have to observe them in every specify analyze job.


Brs,
Bill

On Monday, March 19, 2012 4:39:56 AM UTC+8, Danny Staple wrote:
I suspect the answer here may be to run the child jobs in a build step, while the parent job waits, and then gravs the downstream jobs artifacts via url. The cli can be used for synchronous job runs, and with a newer jenkins you may even be able to use the SSH system to do it (saves grabbing a copy of that cli jar).

You can actually combine those synchronous runs with wgets in a shell script using the same facilities sti has mentioned:

(java -jar jenkins-cli.jar $JENKINS_URL build compile_job && wget <artifact name>) &
(java -jar jenkins-cli.jar $JENKINS_URL build analyze_job && wget <artifact name>) &
wait

I've generalized, and you'll want to look at the cli commands on your server (read https://wiki.jenkins-ci.org/display/JENKINS/Jenkins+CLI) for info on it. I also use a for pid in `jobs` style thing if I've many unix jobs on a shell I want to wait for - again, you'll want to familiarise yourself with the job handling syntax of your shell or build language for such things.

On Friday, 16 March 2012 17:54:27 UTC, sti wrote:
I typically try to run as much in one job as possible to minimize the number of jobs. I use these advanced unix operating systems that allow me to run several things at once like this:

Compile &
Analyze &
wait

If you must split it into multiple jobs, it is possible and even desirable if it allows you to get faster feedback. The only thing is, you cannot send the results upstream. If you have to have a single job where everything is available, both build artifacts and test results, you need to make a new job downstream that collects them both. 

There used to be this option to aggregate test results, which kind of does what you want but I never got it to work. Maybe that option is not compatible with multiconfiguration jobs which in use a lot. 

-- Sami

intelchen <[hidden email]> kirjoitti 16.3.2012 kello 18.03:

Hi,
     It would take so much time to get results of these static code analysis tools.
For example, it takes 12minutes to get findbugs result of our one component. And we have more than 20 components in our products.
And aslo we would like to use checkstyles,pmd besides findbugs.
So I break apart them into two stage jobs. The fist stage is about compliation. The second stage is about these tools.
And these tools could run in parallel.Then we could get the results more quickly,  and send the reports to developers in short time.
   Is it a bad or good practice in Jenkins?

Brs,
Bill

On Friday, March 16, 2012 8:59:13 PM UTC+8, sti wrote:
The most straightforward way is to run the findbug analysis in the same job. Why does it need to be run in its own job?

Jenkins jobs are not as flexible as subroutines in programming languages. If you start using them as such, you will shoot yourself in the foot. 

-- Sami

intelchen <[hidden email]> kirjoitti 16.3.2012 kello 9.28:

Sorry for a uncompleted mail.
And I would like to copy the findbug_result.xml back to the workspaces of the first job.
I think there would be a way to do that is write some shell scripts and copy this result back to the former job.
But is there any other solution or existed plug-ins for this requirement?
May I have your ideas? Thanks!

On Friday, March 16, 2012 3:23:30 PM UTC+8, intelchen wrote:
Hi all,
       There are two stage jobs for one java project.
One is to do compilation job and create a runtime jar file after polling codes from SCM.
Second one is to do findbug analysis job. It copy artifact(runtime jar file) from upstream using Copy Artifact Plugin. And there would be a findbug_result.xml generated after this job.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: How to copy artifacts back to workspaces of upstream job?

danny-orion
In reply to this post by intelchen
First - you'd probably want to redirect output to other files - very large console logs come with problems.
Second - the use of wait (on Unix) should prevent process leaks. Wait is documented in the bash manual.

Thanks,
Danny

On Monday, 19 March 2012 08:46:34 UTC, intelchen wrote:

    If I run several things in background, Jenkins may not capture the output of these commands.
Do you have experiences  about using this  https://wiki.jenkins-ci.org/display/JENKINS/Spawning+processes+from+build ?
   It is easy for me to put everything available in a single job. Then it would be a very large job. It would take a long time to get the results.:( That is my real problem here.
   
On Saturday, March 17, 2012 1:54:27 AM UTC+8, sti wrote:
I typically try to run as much in one job as possible to minimize the number of jobs. I use these advanced unix operating systems that allow me to run several things at once like this:

Compile &
Analyze &
wait

If you must split it into multiple jobs, it is possible and even desirable if it allows you to get faster feedback. The only thing is, you cannot send the results upstream. If you have to have a single job where everything is available, both build artifacts and test results, you need to make a new job downstream that collects them both. 

There used to be this option to aggregate test results, which kind of does what you want but I never got it to work. Maybe that option is not compatible with multiconfiguration jobs which in use a lot. 

-- Sami

intelchen <[hidden email]> kirjoitti 16.3.2012 kello 18.03:

Hi,
     It would take so much time to get results of these static code analysis tools.
For example, it takes 12minutes to get findbugs result of our one component. And we have more than 20 components in our products.
And aslo we would like to use checkstyles,pmd besides findbugs.
So I break apart them into two stage jobs. The fist stage is about compliation. The second stage is about these tools.
And these tools could run in parallel.Then we could get the results more quickly,  and send the reports to developers in short time.
   Is it a bad or good practice in Jenkins?

Brs,
Bill

On Friday, March 16, 2012 8:59:13 PM UTC+8, sti wrote:
The most straightforward way is to run the findbug analysis in the same job. Why does it need to be run in its own job?

Jenkins jobs are not as flexible as subroutines in programming languages. If you start using them as such, you will shoot yourself in the foot. 

-- Sami

intelchen <[hidden email]> kirjoitti 16.3.2012 kello 9.28:

Sorry for a uncompleted mail.
And I would like to copy the findbug_result.xml back to the workspaces of the first job.
I think there would be a way to do that is write some shell scripts and copy this result back to the former job.
But is there any other solution or existed plug-ins for this requirement?
May I have your ideas? Thanks!

On Friday, March 16, 2012 3:23:30 PM UTC+8, intelchen wrote:
Hi all,
       There are two stage jobs for one java project.
One is to do compilation job and create a runtime jar file after polling codes from SCM.
Second one is to do findbug analysis job. It copy artifact(runtime jar file) from upstream using Copy Artifact Plugin. And there would be a findbug_result.xml generated after this job.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: How to copy artifacts back to workspaces of upstream job?

Sami Tikka
In reply to this post by intelchen
I have never had a problem with Jenkins not capturing output. And
AFAICS the the wiki page you refer to does not mention such a problem
either.

Sure, if you want to use background processes in your job, you will
want to make sure you do not exit before you have waited for them all
or killed them off.

If you cancel a job, Jenkins will try to kill your background
processes but it may not be perfect, so do not trust it. (For more
info, search for "Jenkins ProcessTreeKiller")

-- Sami

2012/3/19 intelchen <[hidden email]>:

>
>     If I run several things in background, Jenkins may not capture the
> output of these commands.
> Do you have experiences  about using this
> https://wiki.jenkins-ci.org/display/JENKINS/Spawning+processes+from+build ?
>    It is easy for me to put everything available in a single job. Then it
> would be a very large job. It would take a long time to get the results.:(
> That is my real problem here.
>
> On Saturday, March 17, 2012 1:54:27 AM UTC+8, sti wrote:
>>
>> I typically try to run as much in one job as possible to minimize the
>> number of jobs. I use these advanced unix operating systems that allow me to
>> run several things at once like this:
>>
>> Compile &
>> Analyze &
>> wait
>>
>> If you must split it into multiple jobs, it is possible and even desirable
>> if it allows you to get faster feedback. The only thing is, you cannot send
>> the results upstream. If you have to have a single job where everything is
>> available, both build artifacts and test results, you need to make a new job
>> downstream that collects them both.
>>
>> There used to be this option to aggregate test results, which kind of does
>> what you want but I never got it to work. Maybe that option is not
>> compatible with multiconfiguration jobs which in use a lot.
>>
>> -- Sami
>>
>> intelchen <[hidden email]> kirjoitti 16.3.2012 kello 18.03:
>>
>> Hi,
>>      It would take so much time to get results of these static code
>> analysis tools.
>> For example, it takes 12minutes to get findbugs result of our one
>> component. And we have more than 20 components in our products.
>> And aslo we would like to use checkstyles,pmd besides findbugs.
>> So I break apart them into two stage jobs. The fist stage is about
>> compliation. The second stage is about these tools.
>> And these tools could run in parallel.Then we could get the results more
>> quickly,  and send the reports to developers in short time.
>>    Is it a bad or good practice in Jenkins?
>>
>> Brs,
>> Bill
>>
>> On Friday, March 16, 2012 8:59:13 PM UTC+8, sti wrote:
>>>
>>> The most straightforward way is to run the findbug analysis in the same
>>> job. Why does it need to be run in its own job?
>>>
>>> Jenkins jobs are not as flexible as subroutines in programming languages.
>>> If you start using them as such, you will shoot yourself in the foot.
>>>
>>> -- Sami
>>>
>>> intelchen <[hidden email]> kirjoitti 16.3.2012 kello 9.28:
>>>
>>> Sorry for a uncompleted mail.
>>> And I would like to copy the findbug_result.xml back to the workspaces of
>>> the first job.
>>> I think there would be a way to do that is write some shell scripts and
>>> copy this result back to the former job.
>>> But is there any other solution or existed plug-ins for this requirement?
>>> May I have your ideas? Thanks!
>>>
>>> On Friday, March 16, 2012 3:23:30 PM UTC+8, intelchen wrote:
>>>>
>>>> Hi all,
>>>>        There are two stage jobs for one java project.
>>>> One is to do compilation job and create a runtime jar file after polling
>>>> codes from SCM.
>>>> Second one is to do findbug analysis job. It copy artifact(runtime jar
>>>> file) from upstream using Copy Artifact Plugin. And there would be a
>>>> findbug_result.xml generated after this job.
>>>>
>
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: How to copy artifacts back to workspaces of upstream job?

Jan Seidel-2
Many jobs or not is a kind of religious question XD
I see it exactly contrary to Sami but I suppose this depends also on
the environment ;)
In my department do we have 4 active branches plus 15-20 inactive
which may be revived for maintenance, CRs and the like at any time.
These come along with 50-90 projects and are treated as active
development branches as soon as they are revived. So I favour the
approach of intelchen *bg*

Anyways I have been thinking a bit about your problem and am wondering
if this approach may be of help for your.
You will need the Join plugin for this as minimum requirement. Take
the Copy Artifact Plugin too for convenience; especially when working
with nodes.
http://wiki.jenkins-ci.org/display/JENKINS/Join+Plugin
http://wiki.jenkins-ci.org/display/JENKINS/Copy+Artifact+Plugin

What you do is to run your compile job normally and as post build
action or separate job kick off the test jobs with Join involved here.
The downstream jobs will finish their jobs and notify upstream to Join
which is waiting for all jobs to finish before it comes back to
action.
Join will - when all downstream jobs have finished - trigger a build
that collects the artifacts with Copy Artifact Plugin from each test
job et voilá: you're done.
You should enable fingerprints in Jenkins to make sure that the test
reports ain't mixed up with older reports in case something unexpected
should occur.

Take care
Jan

On 19 Mrz., 18:45, Sami Tikka <[hidden email]> wrote:

> I have never had a problem with Jenkins not capturing output. And
> AFAICS the the wiki page you refer to does not mention such a problem
> either.
>
> Sure, if you want to use background processes in your job, you will
> want to make sure you do not exit before you have waited for them all
> or killed them off.
>
> If you cancel a job, Jenkins will try to kill your background
> processes but it may not be perfect, so do not trust it. (For more
> info, search for "Jenkins ProcessTreeKiller")
>
> -- Sami
>
> 2012/3/19 intelchen <[hidden email]>:
>
>
>
>
>
>
>
>
>
> >     If I run several things in background, Jenkins may not capture the
> > output of these commands.
> > Do you have experiences  about using this
> >https://wiki.jenkins-ci.org/display/JENKINS/Spawning+processes+from+b...?
> >    It is easy for me to put everything available in a single job. Then it
> > would be a very large job. It would take a long time to get the results.:(
> > That is my real problem here.
>
> > On Saturday, March 17, 2012 1:54:27 AM UTC+8, sti wrote:
>
> >> I typically try to run as much in one job as possible to minimize the
> >> number of jobs. I use these advanced unix operating systems that allow me to
> >> run several things at once like this:
>
> >> Compile &
> >> Analyze &
> >> wait
>
> >> If you must split it into multiple jobs, it is possible and even desirable
> >> if it allows you to get faster feedback. The only thing is, you cannot send
> >> the results upstream. If you have to have a single job where everything is
> >> available, both build artifacts and test results, you need to make a new job
> >> downstream that collects them both.
>
> >> There used to be this option to aggregate test results, which kind of does
> >> what you want but I never got it to work. Maybe that option is not
> >> compatible with multiconfiguration jobs which in use a lot.
>
> >> -- Sami
>
> >> intelchen <[hidden email]> kirjoitti 16.3.2012 kello 18.03:
>
> >> Hi,
> >>      It would take so much time to get results of these static code
> >> analysis tools.
> >> For example, it takes 12minutes to get findbugs result of our one
> >> component. And we have more than 20 components in our products.
> >> And aslo we would like to use checkstyles,pmd besides findbugs.
> >> So I break apart them into two stage jobs. The fist stage is about
> >> compliation. The second stage is about these tools.
> >> And these tools could run in parallel.Then we could get the results more
> >> quickly,  and send the reports to developers in short time.
> >>    Is it a bad or good practice in Jenkins?
>
> >> Brs,
> >> Bill
>
> >> On Friday, March 16, 2012 8:59:13 PM UTC+8, sti wrote:
>
> >>> The most straightforward way is to run the findbug analysis in the same
> >>> job. Why does it need to be run in its own job?
>
> >>> Jenkins jobs are not as flexible as subroutines in programming languages.
> >>> If you start using them as such, you will shoot yourself in the foot.
>
> >>> -- Sami
>
> >>> intelchen <[hidden email]> kirjoitti 16.3.2012 kello 9.28:
>
> >>> Sorry for a uncompleted mail.
> >>> And I would like to copy the findbug_result.xml back to the workspaces of
> >>> the first job.
> >>> I think there would be a way to do that is write some shell scripts and
> >>> copy this result back to the former job.
> >>> But is there any other solution or existed plug-ins for this requirement?
> >>> May I have your ideas? Thanks!
>
> >>> On Friday, March 16, 2012 3:23:30 PM UTC+8, intelchen wrote:
>
> >>>> Hi all,
> >>>>        There are two stage jobs for one java project.
> >>>> One is to do compilation job and create a runtime jar file after polling
> >>>> codes from SCM.
> >>>> Second one is to do findbug analysis job. It copy artifact(runtime jar
> >>>> file) from upstream using Copy Artifact Plugin. And there would be a
> >>>> findbug_result.xml generated after this job.
Loading...