maven 2 and downstream builds

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

maven 2 and downstream builds

Stuart Owen-2
Hi,
I have a maven 2 project that is composed of a parent pom and a number of modules. These modules have dependencies between them. I've noticed that whenever I do a build (the goal is install) whenever a module is built it will also trigger the build of other modules that depends upon this. This means that when my job builds some modules can be repeatedly built many times. It also causes other modules in another job to be built even though it depends upon an entirely different module version.


Is there anyway that I can turn off the automatic building of downstream modules? I've had a good look through the configuration options and the documentation but I can't find any mention of how to do this.

By the way, I'm using version 1.106, I also suffer from the Dead(!) executor status with 1.108 thats been mentioned in another thread.

Stuart.




Reply | Threaded
Open this post in threaded view
|

Re: maven 2 and downstream builds

Kohsuke Kawaguchi
Administrator
Stuart Owen wrote:
> Hi,
> I have a maven 2 project that is composed of a parent pom and a  
> number of modules. These modules have dependencies between them. I've  
> noticed that whenever I do a build (the goal is install) whenever a  
> module is built it will also trigger the build of other modules that  
> depends upon this. This means that when my job builds some modules  
> can be repeatedly built many times.

The build triggering in the native m2 project is supposed to analyze the
relatipnship properly so that the same module will not be built more
than once. But sounds like there's a bug in this.

If you can run "hudson.maven.MavenBuild.debug=true;" from the scripting
console, it will print out more details about why a module build is
trigering another. If you can set that and see what's going on, and
perhaps share more details with us, that would be great.

 > It also causes other modules in
> another job to be built even though it depends upon an entirely  
> different module version.

This is being tracked as issue #550. I really need to fix this.


> Is there anyway that I can turn off the automatic building of  
> downstream modules? I've had a good look through the configuration  
> options and the documentation but I can't find any mention of how to  
> do this.

No. I'd rather like to fix this. If you need such option as a
workaround, I'm happy to implement that, but I don't want to make it one
of the officially supported mode.

> By the way, I'm using version 1.106, I also suffer from the Dead(!)  
> executor status with 1.108 thats been mentioned in another thread.

If everything goes well, I'll post 1.109 tomorrow, and that should fix
the problem.

--
Kohsuke Kawaguchi
Sun Microsystems                   [hidden email]

smime.p7s (4K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: maven 2 and downstream builds

Stuart Owen-2

On 8 Jun 2007, at 05:16, Kohsuke Kawaguchi wrote:

Stuart Owen wrote:
Hi,
I have a maven 2 project that is composed of a parent pom and a  number of modules. These modules have dependencies between them. I've  noticed that whenever I do a build (the goal is install) whenever a  module is built it will also trigger the build of other modules that  depends upon this. This means that when my job builds some modules  can be repeatedly built many times.

The build triggering in the native m2 project is supposed to analyze the relatipnship properly so that the same module will not be built more than once. But sounds like there's a bug in this.

If you can run "hudson.maven.MavenBuild.debug=true;" from the scripting console, it will print out more details about why a module build is trigering another. If you can set that and see what's going on, and perhaps share more details with us, that would be great.

Yes, I had this thought after I sent the mail. Maven should order the modules to be built so that those that are depended upon get built first, otherwise of course it wouldn't build at all in a clean environment. What I think may have been happening is that when I build my job, it is rebuilding modules from a previous job. 

I've tested it now but I can't reproduce the fact that for a given build some modules get built many many times. I've been playing around with it a lot and changing various settings so I'm not quite sure what may have caused it, but whatever it is it doesn't seem to be happening any more. I'll keep an eye on it and run the above command if I start seeing this behavior again.


thanks,
Stuart.





Reply | Threaded
Open this post in threaded view
|

Re: maven 2 and downstream builds

Nigel Magnay
I see this too.

FWIW, if everything is building correctly, it builds in what looks
like the correct order. If a module fails, then the build ordering in
subsequent builds seems to go screwy (almost as if it's trying to
build modules that failed last time first).

On 08/06/07, Stuart Owen <[hidden email]> wrote:

>
>
> On 8 Jun 2007, at 05:16, Kohsuke Kawaguchi wrote:
>
> Stuart Owen wrote:
> Hi,
>  I have a maven 2 project that is composed of a parent pom and a  number of
> modules. These modules have dependencies between them. I've  noticed that
> whenever I do a build (the goal is install) whenever a  module is built it
> will also trigger the build of other modules that  depends upon this. This
> means that when my job builds some modules  can be repeatedly built many
> times.
>
> The build triggering in the native m2 project is supposed to analyze the
> relatipnship properly so that the same module will not be built more than
> once. But sounds like there's a bug in this.
>
> If you can run "hudson.maven.MavenBuild.debug=true;" from
> the scripting console, it will print out more details about why a module
> build is trigering another. If you can set that and see what's going on, and
> perhaps share more details with us, that would be great.
> Yes, I had this thought after I sent the mail. Maven should order the
> modules to be built so that those that are depended upon get built first,
> otherwise of course it wouldn't build at all in a clean environment. What I
> think may have been happening is that when I build my job, it is rebuilding
> modules from a previous job.
>
> I've tested it now but I can't reproduce the fact that for a given build
> some modules get built many many times. I've been playing around with it a
> lot and changing various settings so I'm not quite sure what may have caused
> it, but whatever it is it doesn't seem to be happening any more. I'll keep
> an eye on it and run the above command if I start seeing this behavior
> again.
>
>
> thanks,
> Stuart.
>
>
>
>
>
>
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: maven 2 and downstream builds

Stuart Owen-2
Ah yes, this could well explain it. I was getting quite a few build errors during the period I first started noticing it.

On 8 Jun 2007, at 13:28, Nigel Magnay wrote:

I see this too.

FWIW, if everything is building correctly, it builds in what looks
like the correct order. If a module fails, then the build ordering in
subsequent builds seems to go screwy (almost as if it's trying to
build modules that failed last time first).

On 08/06/07, Stuart Owen <[hidden email]> wrote:


On 8 Jun 2007, at 05:16, Kohsuke Kawaguchi wrote:

Stuart Owen wrote:
Hi,
 I have a maven 2 project that is composed of a parent pom and a  number of
modules. These modules have dependencies between them. I've  noticed that
whenever I do a build (the goal is install) whenever a  module is built it
will also trigger the build of other modules that  depends upon this. This
means that when my job builds some modules  can be repeatedly built many
times.

The build triggering in the native m2 project is supposed to analyze the
relatipnship properly so that the same module will not be built more than
once. But sounds like there's a bug in this.

If you can run "hudson.maven.MavenBuild.debug=true;" from
the scripting console, it will print out more details about why a module
build is trigering another. If you can set that and see what's going on, and
perhaps share more details with us, that would be great.
Yes, I had this thought after I sent the mail. Maven should order the
modules to be built so that those that are depended upon get built first,
otherwise of course it wouldn't build at all in a clean environment. What I
think may have been happening is that when I build my job, it is rebuilding
modules from a previous job.

I've tested it now but I can't reproduce the fact that for a given build
some modules get built many many times. I've been playing around with it a
lot and changing various settings so I'm not quite sure what may have caused
it, but whatever it is it doesn't seem to be happening any more. I'll keep
an eye on it and run the above command if I start seeing this behavior
again.


thanks,
Stuart.









---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]


Stuart Owen, myGrid Team
School of Computer Science
The University of Manchester



Reply | Threaded
Open this post in threaded view
|

Re: maven 2 and downstream builds

Kohsuke Kawaguchi
Administrator
Stuart Owen wrote:
> Ah yes, this could well explain it. I was getting quite a few build  
> errors during the period I first started noticing it.

When I implemented this I certainly didn't think about the impact of
failures in a module.

In any case, if you hit a similar issue, please send us your /hudson/log
  which (I think) contains what modules are built in what order.

--
Kohsuke Kawaguchi
Sun Microsystems                   [hidden email]

smime.p7s (4K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: maven 2 and downstream builds

Nigel Magnay
I've just had it do the same - which log file should I look for (there's quite a few in the build...)


On 08/06/07, Kohsuke Kawaguchi <[hidden email]> wrote:
Stuart Owen wrote:
> Ah yes, this could well explain it. I was getting quite a few build
> errors during the period I first started noticing it.

When I implemented this I certainly didn't think about the impact of
failures in a module.

In any case, if you hit a similar issue, please send us your /hudson/log
  which (I think) contains what modules are built in what order.

--
Kohsuke Kawaguchi
Sun Microsystems                   [hidden email]


Reply | Threaded
Open this post in threaded view
|

Re: maven 2 and downstream builds

Kohsuke Kawaguchi
Administrator
Nigel Magnay wrote:
> I've just had it do the same - which log file should I look for (there's
> quite a few in the build...)

   http://yourserver/hudson/log

>
>
> On 08/06/07, Kohsuke Kawaguchi <[hidden email]> wrote:
>>
>> Stuart Owen wrote:
>> > Ah yes, this could well explain it. I was getting quite a few build
>> > errors during the period I first started noticing it.
>>
>> When I implemented this I certainly didn't think about the impact of
>> failures in a module.
>>
>> In any case, if you hit a similar issue, please send us your /hudson/log
>>   which (I think) contains what modules are built in what order.
>>
>> --
>> Kohsuke Kawaguchi
>> Sun Microsystems                   [hidden email]
>>
>>
>

--
Kohsuke Kawaguchi
Sun Microsystems                   [hidden email]

smime.p7s (4K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: maven 2 and downstream builds

Nigel Magnay
Attached: hudson.log showing the builds, and modulebuilds.txt showing some modules being built twice (and the integration 3 times)

HTH
Nigel

On 20/06/07, Kohsuke Kawaguchi <[hidden email]> wrote:
Nigel Magnay wrote:
> I've just had it do the same - which log file should I look for (there's
> quite a few in the build...)

   http://yourserver/hudson/log

>
>
> On 08/06/07, Kohsuke Kawaguchi <[hidden email]> wrote:
>>
>> Stuart Owen wrote:
>> > Ah yes, this could well explain it. I was getting quite a few build
>> > errors during the period I first started noticing it.
>>
>> When I implemented this I certainly didn't think about the impact of
>> failures in a module.
>>
>> In any case, if you hit a similar issue, please send us your /hudson/log
>>   which (I think) contains what modules are built in what order.
>>
>> --
>> Kohsuke Kawaguchi
>> Sun Microsystems                   [hidden email]
>>
>>
>


--
Kohsuke Kawaguchi
Sun Microsystems                   [hidden email]



---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

hudson.log (45K) Download Attachment
modulebuilds.txt (1K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: maven 2 and downstream builds

Nigel Magnay
Further to this - I've occasionally seen successful builds also generate failures.

[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 30 seconds
[INFO] Finished at: Tue Jun 26 17:21:17 BST 2007
[INFO] Final Memory: 27M/49M
[INFO] ------------------------------------------------------------------------
Sending e-mails to: [hidden email]
FATAL: null
java.lang.NullPointerException
at hudson.model.AbstractBuild.getUpstreamRelationship(AbstractBuild.java:314)
at hudson.maven.MavenBuild$RunnerImpl.post (MavenBuild.java:431)
at hudson.model.Run.run(Run.java:572)
at hudson.maven.MavenBuild.run(MavenBuild.java:128)
at hudson.model.Executor.run(Executor.java:59)

Don't know if this is fixed recently, I'm still on 1.106



On 20/06/07, Nigel Magnay <[hidden email]> wrote:
Attached: hudson.log showing the builds, and modulebuilds.txt showing some modules being built twice (and the integration 3 times)

HTH
Nigel


On 20/06/07, Kohsuke Kawaguchi <[hidden email]> wrote:
Nigel Magnay wrote:
> I've just had it do the same - which log file should I look for (there's
> quite a few in the build...)

   <a href="http://yourserver/hudson/log" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)"> http://yourserver/hudson/log

>
>
> On 08/06/07, Kohsuke Kawaguchi <[hidden email]> wrote:
>>
>> Stuart Owen wrote:
>> > Ah yes, this could well explain it. I was getting quite a few build
>> > errors during the period I first started noticing it.
>>
>> When I implemented this I certainly didn't think about the impact of
>> failures in a module.
>>
>> In any case, if you hit a similar issue, please send us your /hudson/log
>>   which (I think) contains what modules are built in what order.
>>
>> --
>> Kohsuke Kawaguchi
>> Sun Microsystems                   [hidden email]
>>
>>
>


--
Kohsuke Kawaguchi
Sun Microsystems                   [hidden email]




Reply | Threaded
Open this post in threaded view
|

Re: maven 2 and downstream builds

Kohsuke Kawaguchi
Administrator

Thanks for the report. Committed a fix. Will be in 1.115.

Nigel Mag nay wrote:

> Further to this - I've occasionally seen successful builds also generate
> failures.
>
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESSFUL
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 30 seconds
> [INFO] Finished at: Tue Jun 26 17:21:17 BST 2007
> [INFO] Final Memory: 27M/49M
> [INFO] ------------------------------------------------------------------------
> Sending e-mails to: [hidden email]
> FATAL: null
> java.lang.NullPointerException
> at hudson.model.AbstractBuild.getUpstreamRelationship(AbstractBuild.java:314)
> at hudson.maven.MavenBuild$RunnerImpl.post(MavenBuild.java:431)
> at hudson.model.Run.run(Run.java:572)
> at hudson.maven.MavenBuild.run(MavenBuild.java:128)
> at hudson.model.Executor.run(Executor.java:59)
>
>
> Don't know if this is fixed recently, I'm still on 1.106
>
>
>
> On 20/06/07, Nigel Magnay <[hidden email]> wrote:
>>
>> Attached: hudson.log showing the builds, and modulebuilds.txt showing some
>> modules being built twice (and the integration 3 times)
>>
>> HTH
>> Nigel
>>
>> On 20/06/07, Kohsuke Kawaguchi <[hidden email]> wrote:
>> >
>> > Nigel Magnay wrote:
>> > > I've just had it do the same - which log file should I look for
>> > (there's
>> > > quite a few in the build...)
>> >
>> >    http://yourserver/hudson/log
>> >
>> > >
>> > >
>> > > On 08/06/07, Kohsuke Kawaguchi <[hidden email]> wrote:
>> > >>
>> > >> Stuart Owen wrote:
>> > >> > Ah yes, this could well explain it. I was getting quite a few build
>> >
>> > >> > errors during the period I first started noticing it.
>> > >>
>> > >> When I implemented this I certainly didn't think about the impact of
>> > >> failures in a module.
>> > >>
>> > >> In any case, if you hit a similar issue, please send us your
>> > /hudson/log
>> > >>   which (I think) contains what modules are built in what order.
>> > >>
>> > >> --
>> > >> Kohsuke Kawaguchi
>> > >> Sun Microsystems                   [hidden email]
>> > >>
>> > >>
>> > >
>> >
>> >
>> > --
>> > Kohsuke Kawaguchi
>> > Sun Microsystems                   [hidden email]
>> >
>> >
>>
>>
>

--
Kohsuke Kawaguchi
Sun Microsystems                   [hidden email]

smime.p7s (4K) Download Attachment