parameterized builds

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

parameterized builds

Tom Huybrechts
I'd like to look at adding the possiblity of parameterized builds.
Features I'm looking for:
- plugins can start a build with parameters
- a way to query the user for parameters: in this case the 'build'
link will not trigger the build but instead ask for the parameters
- allowing multiple builds (with different parameters) in the queue
- persistence of the parameters for queued builds over server restarts
- builder/scm/... configurations should allow references to the build parameters

The biggest problem I see is how AbstractProject == Queue.Task and
AbstractBuild == Queue.Executable.
I'm thinking of allowing plugins to override AbstractProject.doBuild
and createExecutable, and using xstream for queue persistence so we
could persist parameters too.

Thoughts ? Am I making this too complicated / not complicated enough ?

Tom

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: parameterized builds

Kohsuke Kawaguchi
Administrator
Tom Huybrechts wrote:

> I'd like to look at adding the possiblity of parameterized builds.
> Features I'm looking for:
> - plugins can start a build with parameters
> - a way to query the user for parameters: in this case the 'build'
> link will not trigger the build but instead ask for the parameters
> - allowing multiple builds (with different parameters) in the queue
> - persistence of the parameters for queued builds over server restarts
> - builder/scm/... configurations should allow references to the build parameters
>
> The biggest problem I see is how AbstractProject == Queue.Task and
> AbstractBuild == Queue.Executable.
> I'm thinking of allowing plugins to override AbstractProject.doBuild
> and createExecutable, and using xstream for queue persistence so we
> could persist parameters too.
Maybe this notion of parameterized builds should just be in the core,
and the extension point could be just for the different parameter types.

I guess I can't think of any other use of a plugin overriding doBuild.


> Thoughts ? Am I making this too complicated / not complicated enough ?



--
Kohsuke Kawaguchi
Sun Microsystems                   [hidden email]

smime.p7s (4K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

RE: parameterized builds

Mike Salnikov
In reply to this post by Tom Huybrechts
That would be very valuable to have it.
Once it will be possible, Hudson will get closer to our desired feature -
select executor(s) for building on:
https://hudson.dev.java.net/issues/show_bug.cgi?id=1613

----
Mike Salnikov
Senior Developer
Parallels


> -----Original Message-----
> From: Tom Huybrechts [mailto:[hidden email]]
> Sent: Thursday, June 05, 2008 3:50 AM
> To: [hidden email]
> Subject: parameterized builds
>
> I'd like to look at adding the possiblity of parameterized builds.
> Features I'm looking for:
> - plugins can start a build with parameters
> - a way to query the user for parameters: in this case the 'build'
> link will not trigger the build but instead ask for the parameters
> - allowing multiple builds (with different parameters) in the queue
> - persistence of the parameters for queued builds over server restarts
> - builder/scm/... configurations should allow references to the build
> parameters
>
> The biggest problem I see is how AbstractProject == Queue.Task and
> AbstractBuild == Queue.Executable.
> I'm thinking of allowing plugins to override AbstractProject.doBuild
> and createExecutable, and using xstream for queue persistence so we
> could persist parameters too.
>
> Thoughts ? Am I making this too complicated / not complicated enough ?
>
> Tom
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: parameterized builds

Tom Huybrechts
In reply to this post by Kohsuke Kawaguchi
On Thu, Jun 5, 2008 at 3:16 AM, Kohsuke Kawaguchi
<[hidden email]> wrote:

> Tom Huybrechts wrote:
>>
>> I'd like to look at adding the possiblity of parameterized builds.
>> Features I'm looking for:
>> - plugins can start a build with parameters
>> - a way to query the user for parameters: in this case the 'build'
>> link will not trigger the build but instead ask for the parameters
>> - allowing multiple builds (with different parameters) in the queue
>> - persistence of the parameters for queued builds over server restarts
>> - builder/scm/... configurations should allow references to the build
>> parameters
>>
>> The biggest problem I see is how AbstractProject == Queue.Task and
>> AbstractBuild == Queue.Executable.
>> I'm thinking of allowing plugins to override AbstractProject.doBuild
>> and createExecutable, and using xstream for queue persistence so we
>> could persist parameters too.
>
> Maybe this notion of parameterized builds should just be in the core, and
> the extension point could be just for the different parameter types.
>
> I guess I can't think of any other use of a plugin overriding doBuild.
>

I didn't really mean replacing the doBuild method, just allowing the
possiblity to redirect to a separate page to query for parameters...
So this functionality would be in the core, but plugins could contribute pages.

>
>> Thoughts ? Am I making this too complicated / not complicated enough ?
>
>
>
> --
> Kohsuke Kawaguchi
> Sun Microsystems                   [hidden email]
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: parameterized builds

Kohsuke Kawaguchi
Administrator
2008/6/4 Tom Huybrechts <[hidden email]>:

> On Thu, Jun 5, 2008 at 3:16 AM, Kohsuke Kawaguchi
> <[hidden email]> wrote:
>> Tom Huybrechts wrote:
>>>
>>> I'd like to look at adding the possiblity of parameterized builds.
>>> Features I'm looking for:
>>> - plugins can start a build with parameters
>>> - a way to query the user for parameters: in this case the 'build'
>>> link will not trigger the build but instead ask for the parameters
>>> - allowing multiple builds (with different parameters) in the queue
>>> - persistence of the parameters for queued builds over server restarts
>>> - builder/scm/... configurations should allow references to the build
>>> parameters
>>>
>>> The biggest problem I see is how AbstractProject == Queue.Task and
>>> AbstractBuild == Queue.Executable.
>>> I'm thinking of allowing plugins to override AbstractProject.doBuild
>>> and createExecutable, and using xstream for queue persistence so we
>>> could persist parameters too.
>>
>> Maybe this notion of parameterized builds should just be in the core, and
>> the extension point could be just for the different parameter types.
>>
>> I guess I can't think of any other use of a plugin overriding doBuild.
>>
>
> I didn't really mean replacing the doBuild method, just allowing the
> possiblity to redirect to a separate page to query for parameters...
> So this functionality would be in the core, but plugins could contribute pages.

What I meant is that plugins should be allowed to contribute a
fragment of the parameter each, so that the job can aggregate them all
and show it in a single page.

For example, at my work, QA folks wanted to set up a job where they
decide which JDK and which build (we've got multiple branches) of a
project to test, then execute the test. So it would be nice if I can
define them as plugins (so that they can see drop-down for them, not
just the text field), and in their job they nominate those two tings
as the parameters.

In another job, they just need to pick a JDK version to test, so this
job would still choose the former as the parameter but not the latter.

We can then have several general purpose parameter types, like a
free-form text field to an environment variable, etc.


--
Kohsuke Kawaguchi

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: parameterized builds

Tom Huybrechts
On Thu, Jun 5, 2008 at 4:44 PM, Kohsuke Kawaguchi <[hidden email]> wrote:

> 2008/6/4 Tom Huybrechts <[hidden email]>:
>> On Thu, Jun 5, 2008 at 3:16 AM, Kohsuke Kawaguchi
>> <[hidden email]> wrote:
>>> Tom Huybrechts wrote:
>>>>
>>>> I'd like to look at adding the possiblity of parameterized builds.
>>>> Features I'm looking for:
>>>> - plugins can start a build with parameters
>>>> - a way to query the user for parameters: in this case the 'build'
>>>> link will not trigger the build but instead ask for the parameters
>>>> - allowing multiple builds (with different parameters) in the queue
>>>> - persistence of the parameters for queued builds over server restarts
>>>> - builder/scm/... configurations should allow references to the build
>>>> parameters
>>>>
>>>> The biggest problem I see is how AbstractProject == Queue.Task and
>>>> AbstractBuild == Queue.Executable.
>>>> I'm thinking of allowing plugins to override AbstractProject.doBuild
>>>> and createExecutable, and using xstream for queue persistence so we
>>>> could persist parameters too.
>>>
>>> Maybe this notion of parameterized builds should just be in the core, and
>>> the extension point could be just for the different parameter types.
>>>
>>> I guess I can't think of any other use of a plugin overriding doBuild.
>>>
>>
>> I didn't really mean replacing the doBuild method, just allowing the
>> possiblity to redirect to a separate page to query for parameters...
>> So this functionality would be in the core, but plugins could contribute pages.
>
> What I meant is that plugins should be allowed to contribute a
> fragment of the parameter each, so that the job can aggregate them all
> and show it in a single page.
>
> For example, at my work, QA folks wanted to set up a job where they
> decide which JDK and which build (we've got multiple branches) of a
> project to test, then execute the test. So it would be nice if I can
> define them as plugins (so that they can see drop-down for them, not
> just the text field), and in their job they nominate those two tings
> as the parameters.
>
> In another job, they just need to pick a JDK version to test, so this
> job would still choose the former as the parameter but not the latter.
>
> We can then have several general purpose parameter types, like a
> free-form text field to an environment variable, etc.
>

I just committed an initial version to my branch

Provided functionality:
- users can define parameters per project (only string parameters for
now, but parameter types are pluggable)
- if a project has parameters, the 'build' button will redirect to an
input page, where you can provide values for all parameters
- this input page can be overridden to show something fancier, or you
can bypass it and provide the parameters programmatically
- parameter values can be used as ${variable} in Ant, Maven and
CommandInterpreter builders
- after the build, parameter values can be viewed in a separate page
- parameters for parameterized builds are persisted in the queue over restarts
- queue persistence is now in XML (with backwards compatibility for text format)

The UI could use some polishing. In particular I would have liked to
use a popup (hover) for the user input, but HTML is not my exactly my
specialty.
Also defining the parameter definitions felt like reinventing the
wheel - certainly if I would want to add some kind of validation too.
If anyone knows somehting I could reuse here, please let me know.

All other feedback welcome too.

Tom

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

RE: Re: parameterized builds

Michael Donohue-2
RE: Re: parameterized builds

> - if a project has parameters, the 'build' button will redirect to an
> input page, where you can provide values for all parameters

How does this interact with the automated build triggers?  Do they just fail, or use the previously set value of the parameter?


Reply | Threaded
Open this post in threaded view
|

Re: Re: parameterized builds

Tom Huybrechts
On Sun, Jun 8, 2008 at 6:22 PM, Donohue, Michael <[hidden email]> wrote:
>> - if a project has parameters, the 'build' button will redirect to an
>> input page, where you can provide values for all parameters
>
> How does this interact with the automated build triggers?  Do they just
> fail, or use the previously set value of the parameter?
>
>
>

Well, they just build without parameters for the moment.
I think the right behaviour would be to use default values (those are
already configurable).
Only if there are mandatory parameters that do not have a default, the
build would fail.

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: parameterized builds

Kohsuke Kawaguchi
Administrator
Tom Huybrechts wrote:

> On Sun, Jun 8, 2008 at 6:22 PM, Donohue, Michael <[hidden email]> wrote:
>>> - if a project has parameters, the 'build' button will redirect to an
>>> input page, where you can provide values for all parameters
>>
>> How does this interact with the automated build triggers?  Do they just
>> fail, or use the previously set value of the parameter?
>>
>>
>>
>
> Well, they just build without parameters for the moment.
> I think the right behaviour would be to use default values (those are
> already configurable).
> Only if there are mandatory parameters that do not have a default, the
> build would fail.
... or you let parameters to be specified as query parameters or something.

--
Kohsuke Kawaguchi
Sun Microsystems                   [hidden email]

smime.p7s (4K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: parameterized builds

Kohsuke Kawaguchi
Administrator
In reply to this post by Tom Huybrechts

Thanks. I'll take a look at change.

I think I'll understand your comments better once I see it in action.

Tom Huybrechts wrote:

> On Thu, Jun 5, 2008 at 4:44 PM, Kohsuke Kawaguchi <[hidden email]> wrote:
>> 2008/6/4 Tom Huybrechts <[hidden email]>:
>>> On Thu, Jun 5, 2008 at 3:16 AM, Kohsuke Kawaguchi
>>> <[hidden email]> wrote:
>>>> Tom Huybrechts wrote:
>>>>>
>>>>> I'd like to look at adding the possiblity of parameterized builds.
>>>>> Features I'm looking for:
>>>>> - plugins can start a build with parameters
>>>>> - a way to query the user for parameters: in this case the 'build'
>>>>> link will not trigger the build but instead ask for the parameters
>>>>> - allowing multiple builds (with different parameters) in the queue
>>>>> - persistence of the parameters for queued builds over server restarts
>>>>> - builder/scm/... configurations should allow references to the build
>>>>> parameters
>>>>>
>>>>> The biggest problem I see is how AbstractProject == Queue.Task and
>>>>> AbstractBuild == Queue.Executable.
>>>>> I'm thinking of allowing plugins to override AbstractProject.doBuild
>>>>> and createExecutable, and using xstream for queue persistence so we
>>>>> could persist parameters too.
>>>>
>>>> Maybe this notion of parameterized builds should just be in the core, and
>>>> the extension point could be just for the different parameter types.
>>>>
>>>> I guess I can't think of any other use of a plugin overriding doBuild.
>>>>
>>>
>>> I didn't really mean replacing the doBuild method, just allowing the
>>> possiblity to redirect to a separate page to query for parameters...
>>> So this functionality would be in the core, but plugins could contribute pages.
>>
>> What I meant is that plugins should be allowed to contribute a
>> fragment of the parameter each, so that the job can aggregate them all
>> and show it in a single page.
>>
>> For example, at my work, QA folks wanted to set up a job where they
>> decide which JDK and which build (we've got multiple branches) of a
>> project to test, then execute the test. So it would be nice if I can
>> define them as plugins (so that they can see drop-down for them, not
>> just the text field), and in their job they nominate those two tings
>> as the parameters.
>>
>> In another job, they just need to pick a JDK version to test, so this
>> job would still choose the former as the parameter but not the latter.
>>
>> We can then have several general purpose parameter types, like a
>> free-form text field to an environment variable, etc.
>>
>
> I just committed an initial version to my branch
>
> Provided functionality:
> - users can define parameters per project (only string parameters for
> now, but parameter types are pluggable)
> - if a project has parameters, the 'build' button will redirect to an
> input page, where you can provide values for all parameters
> - this input page can be overridden to show something fancier, or you
> can bypass it and provide the parameters programmatically
> - parameter values can be used as ${variable} in Ant, Maven and
> CommandInterpreter builders
> - after the build, parameter values can be viewed in a separate page
> - parameters for parameterized builds are persisted in the queue over restarts
> - queue persistence is now in XML (with backwards compatibility for text format)
>
> The UI could use some polishing. In particular I would have liked to
> use a popup (hover) for the user input, but HTML is not my exactly my
> specialty.
> Also defining the parameter definitions felt like reinventing the
> wheel - certainly if I would want to add some kind of validation too.
> If anyone knows somehting I could reuse here, please let me know.
>
> All other feedback welcome too.
>
> Tom
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>

--
Kohsuke Kawaguchi
Sun Microsystems                   [hidden email]

smime.p7s (4K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: parameterized builds

Tom Huybrechts
In reply to this post by Kohsuke Kawaguchi
On Tue, Jun 10, 2008 at 7:31 PM, Kohsuke Kawaguchi
<[hidden email]> wrote:

> Tom Huybrechts wrote:
>>
>> On Sun, Jun 8, 2008 at 6:22 PM, Donohue, Michael <[hidden email]>
>> wrote:
>>>>
>>>> - if a project has parameters, the 'build' button will redirect to an
>>>> input page, where you can provide values for all parameters
>>>
>>> How does this interact with the automated build triggers?  Do they just
>>> fail, or use the previously set value of the parameter?
>>>
>>>
>>>
>>
>> Well, they just build without parameters for the moment.
>> I think the right behaviour would be to use default values (those are
>> already configurable).
>> Only if there are mandatory parameters that do not have a default, the
>> build would fail.
>
> ... or you let parameters to be specified as query parameters or something.

I had understood that the question referred to builds that trigger each other.
For builds triggered through the remote api, we certainly need query parameters.

>
> --
> Kohsuke Kawaguchi
> Sun Microsystems                   [hidden email]
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]