Quantcast

using wget to download latest artifact

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

using wget to download latest artifact

Gary McWilliams

I am trying to use wget to download the latest build artefact from a server, using the following:

 

wget -P download -r -A gz -I cd http://bfs-pdt-build-1:9090/view/v8/job/8-0-0_nightly/lastStableBuild/artifact/cd

 

If I browse to the URL I can see the files listed, if I examine the html for the index.html on that page the files I need seem to be there (i.e. no “relative” links to the files). But nothing is found.

I’m not a wget expert, but from what I can see in the documentation the above “should” work.

 

Could anyone point me to any info on what I am doing wrong? Can I use wget to download files from hudson?

 



The information in this message is confidential and may be legally privileged. It is intended solely for the addressee. Access to this message by anyone else is unauthorised. If you are not the intended recipient, any disclosure, copying, or distribution of the message, or any action or omission taken by you in reliance on it, is prohibited and may be unlawful. Please immediately contact the sender if you have received this message in error.

The views and opinions expressed in this email may not reflect the views and opinions of any member of Lagan Technologies Limited, or any of its subsidiaries.

Lagan Technologies Limited is a company registered in Northern Ireland with registration number NI 28773. The registered office of Lagan Technologies Limited is 209 Airport Road West, Belfast, Co. Antrim, BT3 9EZ.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: using wget to download latest artifact

Sami Tikka
We use wget on Linux and curl on Mac and they never had any problems.
But we just them very simply:

wget <URL_TO_ARTIFACT_1>
wget <URL_TO_ARTIFACT_2>

For some of our builds, we access the URL where Hudson provides all
the artifacts wrapped in a zip file. (I don't have access to a Hudson
right now so I cannot tell you what that URL is. But it should be easy
to find.)

I never tried to make wget recursively download the artifacts.

If you think Hudson is not behaving properly as a web server, could
you first debug the wget command line against some standard web
server, like Apache?

-- Sami

2009/7/17 Gary McWilliams <[hidden email]>:

> I am trying to use wget to download the latest build artefact from a server,
> using the following:
>
>
>
> wget -P download -r -A gz -I cd
> http://bfs-pdt-build-1:9090/view/v8/job/8-0-0_nightly/lastStableBuild/artifact/cd
>
>
>
> If I browse to the URL I can see the files listed, if I examine the html for
> the index.html on that page the files I need seem to be there (i.e. no
> “relative” links to the files). But nothing is found.
>
> I’m not a wget expert, but from what I can see in the documentation the
> above “should” work.
>
>
>
> Could anyone point me to any info on what I am doing wrong? Can I use wget
> to download files from hudson?
>
>
>
> ________________________________
> The information in this message is confidential and may be legally
> privileged. It is intended solely for the addressee. Access to this message
> by anyone else is unauthorised. If you are not the intended recipient, any
> disclosure, copying, or distribution of the message, or any action or
> omission taken by you in reliance on it, is prohibited and may be unlawful.
> Please immediately contact the sender if you have received this message in
> error.
>
> The views and opinions expressed in this email may not reflect the views and
> opinions of any member of Lagan Technologies Limited, or any of its
> subsidiaries.
>
> Lagan Technologies Limited is a company registered in Northern Ireland with
> registration number NI 28773. The registered office of Lagan Technologies
> Limited is 209 Airport Road West, Belfast, Co. Antrim, BT3 9EZ.
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

RE: using wget to download latest artifact

Gary McWilliams
Hmm, never intended to make any kind of "issue" comment, it's just a request for info/help due to my lack of ability to work this out myself.

My problem is that I do not know what the actual tar.gz file will be called (the built artifact includes the BUILD_ID for example which changes each time), which is why I started down the recursive wget route.

Having said that, I'm not tied to wget. If anyone even knows "another way" of getting the last-successful artifacts, when the name is unknown, then I'm open to suggestions. Using the "all files in zip" seems a step forward, I can see that working now so that's where I'll continue working. Problem is I cannot easily tell if the "latest" available build is a stable/successful build, but I can manage that manually for now.

Thanks.



-----Original Message-----
From: Sami Tikka [mailto:[hidden email]]
Sent: 18 July 2009 20:15
To: [hidden email]
Subject: Re: using wget to download latest artifact

We use wget on Linux and curl on Mac and they never had any problems.
But we just them very simply:

wget <URL_TO_ARTIFACT_1>
wget <URL_TO_ARTIFACT_2>

For some of our builds, we access the URL where Hudson provides all
the artifacts wrapped in a zip file. (I don't have access to a Hudson
right now so I cannot tell you what that URL is. But it should be easy
to find.)

I never tried to make wget recursively download the artifacts.

If you think Hudson is not behaving properly as a web server, could
you first debug the wget command line against some standard web
server, like Apache?

-- Sami

2009/7/17 Gary McWilliams <[hidden email]>:

> I am trying to use wget to download the latest build artefact from a server,
> using the following:
>
>
>
> wget -P download -r -A gz -I cd
> http://bfs-pdt-build-1:9090/view/v8/job/8-0-0_nightly/lastStableBuild/artifact/cd
>
>
>
> If I browse to the URL I can see the files listed, if I examine the html for
> the index.html on that page the files I need seem to be there (i.e. no
> "relative" links to the files). But nothing is found.
>
> I'm not a wget expert, but from what I can see in the documentation the
> above "should" work.
>
>
>
> Could anyone point me to any info on what I am doing wrong? Can I use wget
> to download files from hudson?
>
>
>
> ________________________________
> The information in this message is confidential and may be legally
> privileged. It is intended solely for the addressee. Access to this message
> by anyone else is unauthorised. If you are not the intended recipient, any
> disclosure, copying, or distribution of the message, or any action or
> omission taken by you in reliance on it, is prohibited and may be unlawful.
> Please immediately contact the sender if you have received this message in
> error.
>
> The views and opinions expressed in this email may not reflect the views and
> opinions of any member of Lagan Technologies Limited, or any of its
> subsidiaries.
>
> Lagan Technologies Limited is a company registered in Northern Ireland with
> registration number NI 28773. The registered office of Lagan Technologies
> Limited is 209 Airport Road West, Belfast, Co. Antrim, BT3 9EZ.
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]


The information in this message is confidential and may be legally privileged. It is intended solely for the addressee. Access to this message by anyone else is unauthorised. If you are not the intended recipient, any disclosure, copying, or distribution of the message, or any action or omission taken by you in reliance on it, is prohibited and may be unlawful. Please immediately contact the sender if you have received this message in error.

The views and opinions expressed in this email may not reflect the views and opinions of any member of Lagan Technologies Limited, or any of its subsidiaries.

Lagan Technologies Limited is a company registered in Northern Ireland with registration number NI 28773. The registered office of Lagan Technologies Limited is 209 Airport Road West, Belfast, Co. Antrim, BT3 9EZ.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: using wget to download latest artifact

Sami Tikka
2009/7/20 Gary McWilliams <[hidden email]>:
> Hmm, never intended to make any kind of "issue" comment, it's just a request for info/help due to my lack of ability to work this out myself.

Sorry, I misread you there.

> My problem is that I do not know what the actual tar.gz file will be called (the built artifact includes the BUILD_ID for example which changes each time), which is why I started down the recursive wget route.
>
> Having said that, I'm not tied to wget. If anyone even knows "another way" of getting the last-successful artifacts, when the name is unknown, then I'm open to suggestions. Using the "all files in zip" seems a step forward, I can see that working now so that's where I'll continue working. Problem is I cannot easily tell if the "latest" available build is a stable/successful build, but I can manage that manually for now.

Could you tell us why you cannot tell if the latest available build is
stable or successful? Maybe we could help?

You know the job page contains permalinks with URLs for
lastSuccessfulBuild and lastStableBuild?

Do you have tests that run in a separate Hudson job and the test
result determines if the build is successful or stable?

We have a similar situation in our Hudson. We have solved it like this:

Test job downloads build artifacts from a build job. The test job then
runs the tests and then archives both the test results and the
downloaded artifacts. The stable build is then available from the test
job artifacts.

Another way to do this would be to turn on file fingerprinting and
tell Hudson to aggregate test results to upstream jobs. This should
make it possible to get downstream test job results back to the build
job. We use it for some jobs too but we have found it to be unreliable
with multiconfiguration jobs.

-- Sami

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

RE: using wget to download latest artifact

Gary McWilliams
Yep, background is that there is a downstream job that executes the tests. I'm not using any kind of Maven repo, the test job is just configured to use the same workspace as the main build job.

The "wget" command that I tried to execute was using the "last successful" link, problem remains that I don't know what the artifacts are specifically called, so I'm still stuck on trying to somehow establish the actual names from the download page.

I'll have a look at the aggregation options and see if that helps.


BTW, if you do reply again, but get no further reply from me, please don't think me rude or ignorant of your help. I'll just be away enjoying some sunshine and trying to forget all these kind of problems for a few weeks.



-----Original Message-----
From: Sami Tikka [mailto:[hidden email]]
Sent: 20 July 2009 10:12
To: [hidden email]
Subject: Re: using wget to download latest artifact

Could you tell us why you cannot tell if the latest available build is
stable or successful? Maybe we could help?

You know the job page contains permalinks with URLs for
lastSuccessfulBuild and lastStableBuild?

Do you have tests that run in a separate Hudson job and the test
result determines if the build is successful or stable?

We have a similar situation in our Hudson. We have solved it like this:

Test job downloads build artifacts from a build job. The test job then
runs the tests and then archives both the test results and the
downloaded artifacts. The stable build is then available from the test
job artifacts.

Another way to do this would be to turn on file fingerprinting and
tell Hudson to aggregate test results to upstream jobs. This should
make it possible to get downstream test job results back to the build
job. We use it for some jobs too but we have found it to be unreliable
with multiconfiguration jobs.

-- Sami

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]


The information in this message is confidential and may be legally privileged. It is intended solely for the addressee. Access to this message by anyone else is unauthorised. If you are not the intended recipient, any disclosure, copying, or distribution of the message, or any action or omission taken by you in reliance on it, is prohibited and may be unlawful. Please immediately contact the sender if you have received this message in error.

The views and opinions expressed in this email may not reflect the views and opinions of any member of Lagan Technologies Limited, or any of its subsidiaries.

Lagan Technologies Limited is a company registered in Northern Ireland with registration number NI 28773. The registered office of Lagan Technologies Limited is 209 Airport Road West, Belfast, Co. Antrim, BT3 9EZ.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: using wget to download latest artifact

Richard Bywater
You don't have security enabled or anything like that do you? If you
do, I'd imagine that wget doesn't have the username/password to be
able to access the resources...

Richard.

On 7/21/09, Gary McWilliams <[hidden email]> wrote:

> Yep, background is that there is a downstream job that executes the tests.
> I'm not using any kind of Maven repo, the test job is just configured to use
> the same workspace as the main build job.
>
> The "wget" command that I tried to execute was using the "last successful"
> link, problem remains that I don't know what the artifacts are specifically
> called, so I'm still stuck on trying to somehow establish the actual names
> from the download page.
>
> I'll have a look at the aggregation options and see if that helps.
>
>
> BTW, if you do reply again, but get no further reply from me, please don't
> think me rude or ignorant of your help. I'll just be away enjoying some
> sunshine and trying to forget all these kind of problems for a few weeks.
>
>
>
> -----Original Message-----
> From: Sami Tikka [mailto:[hidden email]]
> Sent: 20 July 2009 10:12
> To: [hidden email]
> Subject: Re: using wget to download latest artifact
>
> Could you tell us why you cannot tell if the latest available build is
> stable or successful? Maybe we could help?
>
> You know the job page contains permalinks with URLs for
> lastSuccessfulBuild and lastStableBuild?
>
> Do you have tests that run in a separate Hudson job and the test
> result determines if the build is successful or stable?
>
> We have a similar situation in our Hudson. We have solved it like this:
>
> Test job downloads build artifacts from a build job. The test job then
> runs the tests and then archives both the test results and the
> downloaded artifacts. The stable build is then available from the test
> job artifacts.
>
> Another way to do this would be to turn on file fingerprinting and
> tell Hudson to aggregate test results to upstream jobs. This should
> make it possible to get downstream test job results back to the build
> job. We use it for some jobs too but we have found it to be unreliable
> with multiconfiguration jobs.
>
> -- Sami
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>
> The information in this message is confidential and may be legally
> privileged. It is intended solely for the addressee. Access to this message
> by anyone else is unauthorised. If you are not the intended recipient, any
> disclosure, copying, or distribution of the message, or any action or
> omission taken by you in reliance on it, is prohibited and may be unlawful.
> Please immediately contact the sender if you have received this message in
> error.
>
> The views and opinions expressed in this email may not reflect the views and
> opinions of any member of Lagan Technologies Limited, or any of its
> subsidiaries.
>
> Lagan Technologies Limited is a company registered in Northern Ireland with
> registration number NI 28773. The registered office of Lagan Technologies
> Limited is 209 Airport Road West, Belfast, Co. Antrim, BT3 9EZ.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Loading...