RE: Re: Failed builds triggering downstream builds

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

RE: Re: Failed builds triggering downstream builds

Matt Tucker-4
(Moved to dev@ because it's seems like we're getting into a lot of
detail here)

Kohsuke Kawaguchi wrote:

> Matt Tucker wrote:
>> Kohsuke Kawaguchi wrote:
>>> Matt Tucker wrote:
>>>> Kohsuke Kawaguchi wrote:
>>>>> Matt Tucker wrote:
>>>> I don't see anything in the build output or console that looks like
>>>> it's triggering downstream builds.
>>>
>>> Then I suspect those downstream builds are not triggered by the
>>> upstream.
>>
>> I'm nearly certain they are, I just couldn't find the console
>> output. If B,C,D depends on A, and I build A, and as soon as it's
>> done building B,C,D start building, that kind of says to me that
>> something other than the SCM is triggering the build (which would be
>> the alternative).
>>
>> Sounds, though, like I'm going to have to prove it before you believe
>> me.  :)
>
> No, that's not what I meant, but we should first look at why there's
> no information left in the upstream.
>
> What is the job type of the upstream?

I'm using a custom job type that only allows the user to declare a
project name and some basic details (such as svn branch name).
Everything else (such as adding particuarly publishers, build steps,
etc.) is configured automatically by the custom job type.

In a way it's like a freestyle build that's automatically configured by
code.


> When you see A's configuration, does it list B,C, and D as the
> dependency projects?

Not in the configuration, no.  I do see the upstream/downstream info
from fingerprints.  But the dependencies for building are all calculated
dynamically by the IvyBuildTrigger, which registers as a
"DependecyDeclarer" (looks like a typo there).

IvyBuildTrigger has (essentially):

    public boolean perform(Build<?, ?> build, Launcher launcher,
BuildListener listener) {
        . . .
            if (lastmodified != f.lastModified()) {
 
recomputeModuleDescriptor(build.getProject().getWorkspace());
                Hudson.getInstance().rebuildDependencyGraph();
            }
        return true;
    }

and (pseudo-code):

    public void buildDependencyGraph(AbstractProject owner,
DependencyGraph graph) {
        for each dependency of owner:
            map to an existing Hudson project
            if map was successful:
               add to list of owner's dependencies
}

None of this stuff gets saved when the project's configuration is saved,
and it doesn't affect the manually maintained list of project
dependencies, either up- or down-stream.  I notice in the saved
config.xml for one of these projects:

  <keepDependencies>false</keepDependencies>

which implies to me that these dependencies are only maintained in
memory.

One thing I find interesting about DependencyGraph is this comment:

 * Once built, {@link DependencyGraph} is immutable, and every time
 * there's a change (which is relatively rare), a new instance
 * will be created. This eliminates the need of synchronization.

This is not true for IvyBuildTrigger, which causes a dependency graph
rebuild after every build. Presumably this is fine, and I can't see
where it would cause the behavior that I'm seeing, but I think it's
interesting that using automatically generated build dependencies
invalidates this assumption of near-immutability.

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

AW: Re: Failed builds triggering downstream builds

Martin Ficker
I havent followed your previous discussion, but:
> This is not true for IvyBuildTrigger, which causes a dependency graph
rebuild after every build.  
That because the build (or scm checkout) might have changed the ivy file.

-----Urspr√ľngliche Nachricht-----
Von: Matt Tucker [mailto:[hidden email]]
Gesendet: Mittwoch, 2. Juli 2008 01:32
An: [hidden email]
Betreff: RE: Re: Failed builds triggering downstream builds

(Moved to dev@ because it's seems like we're getting into a lot of detail
here)

Kohsuke Kawaguchi wrote:

> Matt Tucker wrote:
>> Kohsuke Kawaguchi wrote:
>>> Matt Tucker wrote:
>>>> Kohsuke Kawaguchi wrote:
>>>>> Matt Tucker wrote:
>>>> I don't see anything in the build output or console that looks like
>>>> it's triggering downstream builds.
>>>
>>> Then I suspect those downstream builds are not triggered by the
>>> upstream.
>>
>> I'm nearly certain they are, I just couldn't find the console output.
>> If B,C,D depends on A, and I build A, and as soon as it's done
>> building B,C,D start building, that kind of says to me that something
>> other than the SCM is triggering the build (which would be the
>> alternative).
>>
>> Sounds, though, like I'm going to have to prove it before you believe
>> me.  :)
>
> No, that's not what I meant, but we should first look at why there's
> no information left in the upstream.
>
> What is the job type of the upstream?

I'm using a custom job type that only allows the user to declare a project
name and some basic details (such as svn branch name).
Everything else (such as adding particuarly publishers, build steps,
etc.) is configured automatically by the custom job type.

In a way it's like a freestyle build that's automatically configured by
code.


> When you see A's configuration, does it list B,C, and D as the
> dependency projects?

Not in the configuration, no.  I do see the upstream/downstream info from
fingerprints.  But the dependencies for building are all calculated
dynamically by the IvyBuildTrigger, which registers as a "DependecyDeclarer"
(looks like a typo there).

IvyBuildTrigger has (essentially):

    public boolean perform(Build<?, ?> build, Launcher launcher,
BuildListener listener) {
        . . .
            if (lastmodified != f.lastModified()) {
 
recomputeModuleDescriptor(build.getProject().getWorkspace());
                Hudson.getInstance().rebuildDependencyGraph();
            }
        return true;
    }

and (pseudo-code):

    public void buildDependencyGraph(AbstractProject owner, DependencyGraph
graph) {
        for each dependency of owner:
            map to an existing Hudson project
            if map was successful:
               add to list of owner's dependencies }

None of this stuff gets saved when the project's configuration is saved, and
it doesn't affect the manually maintained list of project dependencies,
either up- or down-stream.  I notice in the saved config.xml for one of
these projects:

  <keepDependencies>false</keepDependencies>

which implies to me that these dependencies are only maintained in memory.

One thing I find interesting about DependencyGraph is this comment:

 * Once built, {@link DependencyGraph} is immutable, and every time
 * there's a change (which is relatively rare), a new instance
 * will be created. This eliminates the need of synchronization.

This is not true for IvyBuildTrigger, which causes a dependency graph
rebuild after every build. Presumably this is fine, and I can't see where it
would cause the behavior that I'm seeing, but I think it's interesting that
using automatically generated build dependencies invalidates this assumption
of near-immutability.

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]



---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

RE: AW: Re: Failed builds triggering downstream builds

Matt Tucker-4
Martin Eigenbrodt wrote:
> I havent followed your previous discussion, but:
>> This is not true for IvyBuildTrigger, which causes a dependency graph
>> rebuild after every build.
>
> That because the build (or scm checkout) might have changed the ivy
> file.

Oh, I totally understand why it's happening, and it's completely the
right thing to do.  The primary flaw that I was trying to point out is
the assumption that the dependency graph is largely immutable, when in
fact in cases where you're calculating it rather than trying to maintain
it manually, it could easily change with any given checkin (although it
generally doesn't).

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]