Distributed Builds: multiple native projects in multiple platforms?

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Distributed Builds: multiple native projects in multiple platforms?

Thiago Bastos
Hello,

I currently use CruiseControl for continuous integration of several C++ projects, and am highly inclined to switch to Hudson for its elegant design and support for distributed builds. However I am not sure what would be the best way to setup Hudson in our environment.

Most of our projects are cross-platform, so we have a small set of heterogeneous build servers (mostly Windows and Linux). With CruiseControl we had to setup a CC instance per host, but with Hudson I hope to have a centralized place to monitor all projects in all platforms. Basically I'd have only one master Hudson, and then several slaves running in different platforms. But how can I guarantee that every project will be built and tested by at least one slave in each platform?

The main question is, do I have to setup one job per platform for each project (say, project1_win32, project1_linux26_i386, etc), or is there a better approach? In this case I would have to stick each job to a slave running the corresponding platform. I guess there is no way to stick a job to a "set of slaves" (in case I have multiple slaves running the same platform)?

PS: wow the website turned into Confluence while I was browsing the docs! Nice to know you use Confluence! Are there any plans for a Hudson/Confluence integration plugin? Sorry, couldn't resist... :-)

Thanks a lot!
-- Thiago
Reply | Threaded
Open this post in threaded view
|

Re: Distributed Builds: multiple native projects in multiple platforms?

Arnaud LACOUR
On 5/6/07, Thiago Bastos <[hidden email]> wrote:

> Hello,
>
> I currently use CruiseControl for continuous integration of several C++
> projects, and am highly inclined to switch to Hudson for its elegant design
> and support for distributed builds. However I am not sure what would be the
> best way to setup Hudson in our environment.
>
> Most of our projects are cross-platform, so we have a small set of
> heterogeneous build servers (mostly Windows and Linux). With CruiseControl
> we had to setup a CC instance per host, but with Hudson I hope to have a
> centralized place to monitor all projects in all platforms. Basically I'd
> have only one master Hudson, and then several slaves running in different
> platforms. But how can I guarantee that every project will be built and
> tested by at least one slave in each platform?
You can tie a job a to slave.
There's no support of distribution per platform right now, so you
would have to handle the distribution in your build scripts for
example.
>
> The main question is, do I have to setup one job per platform for each
> project (say, project1_win32, project1_linux26_i386, etc), or is there a
> better approach?
That's the approach I have taken.
> In this case I would have to stick each job to a slave
> running the corresponding platform. I guess there is no way to stick a job
> to a "set of slaves" (in case I have multiple slaves running the same
> platform)?
no
That could be a nice extension though
>
> PS: wow the website turned into Confluence while I was browsing the docs!
> Nice to know you use Confluence! Are there any plans for a Hudson/Confluence
> integration plugin? Sorry, couldn't resist... :-)
>
> Thanks a lot!
> -- Thiago
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Distributed Builds: multiple native projects in multiple platforms?

Wolfram Kroll-2
Arnaud LACOUR schrieb:

> On 5/6/07, Thiago Bastos <[hidden email]> wrote:
>> Hello,
>>
>> I currently use CruiseControl for continuous integration of several C++
>> projects, and am highly inclined to switch to Hudson for its elegant
>> design
>> and support for distributed builds. However I am not sure what would
>> be the
>> best way to setup Hudson in our environment.
>>
>> Most of our projects are cross-platform, so we have a small set of
>> heterogeneous build servers (mostly Windows and Linux). With
>> CruiseControl
>> we had to setup a CC instance per host, but with Hudson I hope to have a
>> centralized place to monitor all projects in all platforms. Basically I'd
>> have only one master Hudson, and then several slaves running in different
>> platforms. But how can I guarantee that every project will be built and
>> tested by at least one slave in each platform?
> You can tie a job a to slave.
> There's no support of distribution per platform right now, so you
> would have to handle the distribution in your build scripts for
> example.
>>
>> The main question is, do I have to setup one job per platform for each
>> project (say, project1_win32, project1_linux26_i386, etc), or is there a
>> better approach?
> That's the approach I have taken.
>> In this case I would have to stick each job to a slave
>> running the corresponding platform. I guess there is no way to stick a
>> job
>> to a "set of slaves" (in case I have multiple slaves running the same
>> platform)?
> no
> That could be a nice extension though

I suggest an enhancement: having the possibility to set properties on
build machines. Having the same value of a property would form a group
of machines. Then I could tell hudson "build this job on a slave of
group 'win32_xyz'"! For your use case you'd still have to define 2 jobs,
one for each platform group.

I was asked to do this as a plugin but I don't have time for this at
work (not Google - no 20% projects ;-)

Wolfram

>>
>> PS: wow the website turned into Confluence while I was browsing the docs!
>> Nice to know you use Confluence! Are there any plans for a
>> Hudson/Confluence
>> integration plugin? Sorry, couldn't resist... :-)
>>
>> Thanks a lot!
>> -- Thiago
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Distributed Builds: multiple native projects in multiple platforms?

Kohsuke Kawaguchi
Administrator
In reply to this post by Thiago Bastos
2007/5/6, Thiago Bastos <[hidden email]>:
> The main question is, do I have to setup one job per platform for each
> project (say, project1_win32, project1_linux26_i386, etc), or is there a
> better approach? In this case I would have to stick each job to a slave
> running the corresponding platform. I guess there is no way to stick a job
> to a "set of slaves" (in case I have multiple slaves running the same
> platform)?

Hudson has two problems in this area.

1. How Hudson allocates jobs to slaves is not very clever. I really
need to extend this so that you can say "this job runs on slave with
tag X", etc. I think there's an RFE filed for that, but maybe not. I
need this for myself for my production system, so it'll probably get
implemented soon.

2. Hudson needs to allow you to create a single configuration and run
it on multiple "platforms" (in the loose sense of the word.) Our
testing team needs this so that they can run tests on different
platforms, in their case multiple OS, JDK versions, application
servers, etc.

Right now you need to create new job for each different configuration
combination, and when they build, their build numbers can get out of
sync. So this really needs to be improved too. And again,


> PS: wow the website turned into Confluence while I was browsing the docs!
> Nice to know you use Confluence! Are there any plans for a Hudson/Confluence
> integration plugin? Sorry, couldn't resist... :-)

What kind of integration would you like to see in the plugin?

--
Kohsuke Kawaguchi

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Distributed Builds: multiple native projects in multiple platforms?

Kohsuke Kawaguchi
Administrator
In reply to this post by Arnaud LACOUR
2007/5/6, Arnaud LACOUR <[hidden email]>:
> You can tie a job a to slave.
> There's no support of distribution per platform right now, so you
> would have to handle the distribution in your build scripts for
> example.

I'm thinking that allowing slaves to have tags, then allowing jobs to
be assigned to tags seem like a reasonable option.

Hudson can put several default tags by doing a bit of auto-detection
based on system properties, like OS, CPU architecture, etc.

--
Kohsuke Kawaguchi

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Distributed Builds: multiple native projects in multiple platforms?

Thiago Bastos
In reply to this post by Kohsuke Kawaguchi
2007/5/8, Kohsuke Kawaguchi <[hidden email]>:
Hudson has two problems in this area.

1. How Hudson allocates jobs to slaves is not very clever. I really
need to extend this so that you can say "this job runs on slave with
tag X", etc. I think there's an RFE filed for that, but maybe not. I
need this for myself for my production system, so it'll probably get
implemented soon.

2. Hudson needs to allow you to create a single configuration and run
it on multiple "platforms" (in the loose sense of the word.) Our
testing team needs this so that they can run tests on different
platforms, in their case multiple OS, JDK versions, application
servers, etc.

Right now you need to create new job for each different configuration
combination, and when they build, their build numbers can get out of
sync. So this really needs to be improved too. And again,

It's really GREAT to hear you have those improvements in mind!

> PS: wow the website turned into Confluence while I was browsing the docs!
> Nice to know you use Confluence! Are there any plans for a Hudson/Confluence
> integration plugin? Sorry, couldn't resist... :-)

What kind of integration would you like to see in the plugin?

Well, the above-mentioned matters are much more important :-)
But I was thinking about something in the likes of Bamboo, where you can embed "fragments of Bamboo content" into HTML pages. In Bamboo they offer "Javascript widgets", so it's not really Confluence-specific. You can easily embed tables & reports into any HTML page. I'd say it's a nice feature to have, but not really a priority.

Thanks!
Thiago

Reply | Threaded
Open this post in threaded view
|

Re: Distributed Builds: multiple native projects in multiple platforms?

Wolfram Kroll-2
In reply to this post by Kohsuke Kawaguchi

Am 08.05.2007 um 15:53 schrieb Kohsuke Kawaguchi:

> 2007/5/6, Arnaud LACOUR <[hidden email]>:
>> You can tie a job a to slave.
>> There's no support of distribution per platform right now, so you
>> would have to handle the distribution in your build scripts for
>> example.
>
> I'm thinking that allowing slaves to have tags, then allowing jobs to
> be assigned to tags seem like a reasonable option.

I love to hear that!

>
> Hudson can put several default tags by doing a bit of auto-detection
> based on system properties, like OS, CPU architecture, etc.

Even better! When yo implement this, remember there must be some UI  
to select tags when I create a job.

Wolfram

>
> --
> Kohsuke Kawaguchi
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Distributed Builds: multiple native projects in multiple platforms?

Kohsuke Kawaguchi
Administrator
In reply to this post by Thiago Bastos
2007/5/8, Thiago Bastos <[hidden email]>:

> > > integration plugin? Sorry, couldn't resist... :-)
> >
> > What kind of integration would you like to see in the plugin?
>
>
> Well, the above-mentioned matters are much more important :-)
> But I was thinking about something in the likes of Bamboo, where you can
> embed "fragments of Bamboo content" into HTML pages. In Bamboo they offer
> "Javascript widgets", so it's not really Confluence-specific. You can easily
> embed tables & reports into any HTML page. I'd say it's a nice feature to
> have, but not really a priority.

Since Hudson remote API supports JSON, this is entirely feasible both
as JavaScript outside Hudson or as a plugin.

If anyone is interested in working on it, that would be great.

--
Kohsuke Kawaguchi

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]