Feedback on the matrix plugin

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

Feedback on the matrix plugin

Rajesh Rajasekaran-4
I found the matrix plugin to suit a lot of our testing needs and decided
to test a simple matrix, build a project with 2 jdk's.

Here are some of the observations on the behavior of the plugin, though
i couldn't realize if these were bugs or the intended behavior. I
understand that the plugin is in its early phases, and thought early
feedback would be helpful.

I had selected a label with a pool of slaves for this build.

1. The matrix project chooses a slave from outside the pool and uses it
to check out the source, and then triggers the 2 configurations on the
slave pool i selected.
What is the purpose of the initial job, to trigger the configurations?
if so why does it check out the source, because when my 2 configurations
are built they check out the source again.
Also now, three of my slaves are occupied until this build is completed,
the slave used for the initial trigger remains just idle.

2. For the initial job mentioned above, hudson randomly chooses a slave
which is configured with a usage "Utilize this slave as much as
possible". I i have no slave with this usage, the matrix job never executes.

3. It would be nice if we could have the name of the job in the email
subject, followed my the config name. Right now, only the cofig name is
added to the email subject.
Hudson build became unstable: jdk=java15,label=linux

4. At present, we have a maximum need for 8 configurations in the
matrix, wher each of the config would take upto 5 hrs to run. It would
be nice if i could run any one of the config separately. Right now, the
"Build Now" option exists only the top level project. I understand that
this is right against the purpose of the matrix, but for time consuming
builds this would be helpful.

5. We would like a have a matrix build which would build the project on
different architectures, OS's with jdk5. I have selected a slave for
each architecture in my build. But the jdk5 used for each architecture
is in a different location, like /tools/x86_64/jdk1.5,
/tools/ia64/jdk1.5. How do i tell the matrix to use a particular jdk
when building on a particular arch.

Any feedback on the above is much appreciated.

Thanks
Rajesh

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Feedback on the matrix plugin

Stephan Zeissler (KUTTIG)


Rajesh Rajasekaran wrote:
> 5. We would like a have a matrix build which would build the project
> on different architectures, OS's with jdk5. I have selected a slave
> for each architecture in my build. But the jdk5 used for each
> architecture is in a different location, like /tools/x86_64/jdk1.5,
> /tools/ia64/jdk1.5. How do i tell the matrix to use a particular jdk
> when building on a particular arch.
AFAIK the hudson slave model currently requires you to have a common
layout on all slaves (which is understandable to me but frustrating for
the initial setup). There was an introduction article for hudson slaves
somewhere ... (found it: [1]).
Maybe you can start by placing symbolic links in the hudson slaves home
and use them as the jdk path.

@koshuke: Isn't it possible to add a configuration option in the slave
config page so you can overwrite the paths for jdk, ant and maven for
specific slaves?

-Stephan

[1] http://hudson.gotdns.com/wiki/display/HUDSON/Distributed+builds

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

stephan.zeissler.vcf (381 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Feedback on the matrix plugin

Kohsuke Kawaguchi
Administrator
In reply to this post by Rajesh Rajasekaran-4
Rajesh Rajasekaran wrote:
> I found the matrix plugin to suit a lot of our testing needs and decided
> to test a simple matrix, build a project with 2 jdk's.

Thanks for the feedback.

> Here are some of the observations on the behavior of the plugin, though
> i couldn't realize if these were bugs or the intended behavior. I
> understand that the plugin is in its early phases, and thought early
> feedback would be helpful.
>
> I had selected a label with a pool of slaves for this build.
>
> 1. The matrix project chooses a slave from outside the pool and uses it
> to check out the source, and then triggers the 2 configurations on the
> slave pool i selected.
> What is the purpose of the initial job, to trigger the configurations?
> if so why does it check out the source, because when my 2 configurations
> are built they check out the source again.
Yeah, the parent job checking out the source code is redundant. I'm just
doing it so that I can get a changelog and workspace displayed, but I'm
sure there's a better way to do this.

> Also now, three of my slaves are occupied until this build is completed,
> the slave used for the initial trigger remains just idle.

Yes. This was again just because it was easy to do it this way. The
parent build also serves as a lock, so that all configurations are built
before a next iteration starts.

It would be better if this can be done outside normal executors. There
are several other things in this category, like cvs/svn tagging
operation that runs outside executors.


> 2. For the initial job mentioned above, hudson randomly chooses a slave
> which is configured with a usage "Utilize this slave as much as
> possible". I i have no slave with this usage, the matrix job never executes.

I guess fixing the above would solve the problem.

> 3. It would be nice if we could have the name of the job in the email
> subject, followed my the config name. Right now, only the cofig name is
> added to the email subject.
> Hudson build became unstable: jdk=java15,label=linux

I believe this was fixed in 1.131.

> 4. At present, we have a maximum need for 8 configurations in the
> matrix, wher each of the config would take upto 5 hrs to run. It would
> be nice if i could run any one of the config separately. Right now, the
> "Build Now" option exists only the top level project. I understand that
> this is right against the purpose of the matrix, but for time consuming
> builds this would be helpful.

Yeah, I agree. Sometimes you'd need to be able to just rerun this on a
subset would be useful.

> 5. We would like a have a matrix build which would build the project on
> different architectures, OS's with jdk5. I have selected a slave for
> each architecture in my build. But the jdk5 used for each architecture
> is in a different location, like /tools/x86_64/jdk1.5,
> /tools/ia64/jdk1.5. How do i tell the matrix to use a particular jdk
> when building on a particular arch.

Stephan already answered this point. I do this by using symlinks.


--
Kohsuke Kawaguchi
Sun Microsystems                   [hidden email]

smime.p7s (4K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Feedback on the matrix plugin

Kohsuke Kawaguchi
Administrator
In reply to this post by Stephan Zeissler (KUTTIG)
Stephan Zeissler (KUTTIG) wrote:
> @koshuke: Isn't it possible to add a configuration option in the slave
> config page so you can overwrite the paths for jdk, ant and maven for
> specific slaves?

Yes. This problem is particularly bad on Windows vs Unix.

Doing this in the system config screen is bit hard. When a new JDK is
added, I'd have to also update the other parts of the form to include
new entries.

I think it's time to have per-slave config page, and we can add stuff
like this there.

--
Kohsuke Kawaguchi
Sun Microsystems                   [hidden email]

smime.p7s (4K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Feedback on the matrix plugin

Stephan Zeissler (KUTTIG)

Kohsuke Kawaguchi schrieb:

> Stephan Zeissler (KUTTIG) wrote:
>> @koshuke: Isn't it possible to add a configuration option in the
>> slave config page so you can overwrite the paths for jdk, ant and
>> maven for specific slaves?
>
> Yes. This problem is particularly bad on Windows vs Unix.
>
> Doing this in the system config screen is bit hard. When a new JDK is
> added, I'd have to also update the other parts of the form to include
> new entries.
>
> I think it's time to have per-slave config page, and we can add stuff
> like this there.
Yes, I meant a config page per slave. Isn't there already one?

- Stephan

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

stephan.zeissler.vcf (381 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

RE: Re: Feedback on the matrix plugin

ruel loehr
In reply to this post by Kohsuke Kawaguchi
+1.

After moving from Hudson from Cruisecontrol I can say it is WAY easier
to use.  The only gotcha that I ran into was having to do a hack to get
my solaris slave to run from the windows master.  

The hack was simply to call a shell script as part of my ssh login on
slave connection and to set my ant home and such.   It was confusing at
the time.....



-----Original Message-----
From: Kohsuke Kawaguchi [mailto:[hidden email]]
Sent: Monday, August 27, 2007 8:31 PM
To: [hidden email]
Subject: Re: Feedback on the matrix plugin

Stephan Zeissler (KUTTIG) wrote:
> @koshuke: Isn't it possible to add a configuration option in the slave

> config page so you can overwrite the paths for jdk, ant and maven for
> specific slaves?

Yes. This problem is particularly bad on Windows vs Unix.

Doing this in the system config screen is bit hard. When a new JDK is
added, I'd have to also update the other parts of the form to include
new entries.

I think it's time to have per-slave config page, and we can add stuff
like this there.

--
Kohsuke Kawaguchi
Sun Microsystems                   [hidden email]

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Feedback on the matrix plugin

Rajesh Rajasekaran-4
In reply to this post by Kohsuke Kawaguchi
Thanks for the reply.

>
> Yeah, the parent job checking out the source code is redundant. I'm
> just doing it so that I can get a changelog and workspace displayed,
> but I'm sure there's a better way to do this.
>

Maybe, we can devote a slave with more number of executors to handle
this job for all the matrix projects we use.


>> 3. It would be nice if we could have the name of the job in the email
>> subject, followed my the config name. Right now, only the cofig name
>> is added to the email subject.
>> Hudson build became unstable: jdk=java15,label=linux
>
> I believe this was fixed in 1.131.
>

Thanks. I will upgrade to this.

>> 4. At present, we have a maximum need for 8 configurations in the
>> matrix, wher each of the config would take upto 5 hrs to run. It
>> would be nice if i could run any one of the config separately. Right
>> now, the "Build Now" option exists only the top level project. I
>> understand that this is right against the purpose of the matrix, but
>> for time consuming builds this would be helpful.
>
> Yeah, I agree. Sometimes you'd need to be able to just rerun this on a
> subset would be useful.
>

Yes. this feature would be nice.

>> 5. We would like a have a matrix build which would build the project
>> on different architectures, OS's with jdk5. I have selected a slave
>> for each architecture in my build. But the jdk5 used for each
>> architecture is in a different location, like /tools/x86_64/jdk1.5,
>> /tools/ia64/jdk1.5. How do i tell the matrix to use a particular jdk
>> when building on a particular arch.
>
> Stephan already answered this point. I do this by using symlinks.
>
>

Thanks Stephan for the input.
However symlinks won't work for us. We use shared nfs for all the
slaves. But I think we can handle the jdk's in our build script based on
the host.

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Feedback on the matrix plugin

Kohsuke Kawaguchi
Administrator
Rajesh Rajasekaran wrote:

> Thanks for the reply.
>
>>
>> Yeah, the parent job checking out the source code is redundant. I'm
>> just doing it so that I can get a changelog and workspace displayed,
>> but I'm sure there's a better way to do this.
>>
>
> Maybe, we can devote a slave with more number of executors to handle
> this job for all the matrix projects we use.
Yes, that would be a work around in the mean time.


>>> 5. We would like a have a matrix build which would build the project
>>> on different architectures, OS's with jdk5. I have selected a slave
>>> for each architecture in my build. But the jdk5 used for each
>>> architecture is in a different location, like /tools/x86_64/jdk1.5,
>>> /tools/ia64/jdk1.5. How do i tell the matrix to use a particular jdk
>>> when building on a particular arch.
>>
>> Stephan already answered this point. I do this by using symlinks.
>>
>>
>
> Thanks Stephan for the input.
> However symlinks won't work for us. We use shared nfs for all the
> slaves. But I think we can handle the jdk's in our build script based on
> the host.
Are your slaves completely diskless? Otherwise you should be able to
still define symlinks.

--
Kohsuke Kawaguchi
Sun Microsystems                   [hidden email]

smime.p7s (4K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Feedback on the matrix plugin

Kohsuke Kawaguchi
Administrator
In reply to this post by Stephan Zeissler (KUTTIG)
Stephan Zeissler (KUTTIG) wrote:
>> I think it's time to have per-slave config page, and we can add stuff
>> like this there.
 >
> Yes, I meant a config page per slave. Isn't there already one?

No.

If you can file this (per slave path setting) as RFE in the issue
tracker, that would be great.

--
Kohsuke Kawaguchi
Sun Microsystems                   [hidden email]

smime.p7s (4K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Feedback on the matrix plugin

Stephan Zeissler (KUTTIG)
https://hudson.dev.java.net/issues/show_bug.cgi?id=786

Kohsuke Kawaguchi schrieb:

> Stephan Zeissler (KUTTIG) wrote:
>>> I think it's time to have per-slave config page, and we can add
>>> stuff like this there.
> >
>> Yes, I meant a config page per slave. Isn't there already one?
>
> No.
>
> If you can file this (per slave path setting) as RFE in the issue
> tracker, that would be great.
>
--
Stephan Zeissler
Software-Development

KUTTIG Computeranwendungen GmbH
Frankfurter Stra├če 35
53840 Troisdorf
MOB +49 (173) 7207900
FON +49 (2241) 9833-413
FAX +49 (2241) 9833-100
EMAIL [hidden email]
WEB www.kuttig.com
GF Dipl.-Kfm. Klaus Kuttig
Michael Wessels
HR Siegburg Nr 2848


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

stephan.zeissler.vcf (381 bytes) Download Attachment