Multiple archiving for maven-style projects

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Multiple archiving for maven-style projects

Seva Popov
Hi,

Could somebody explain why Hudson executes the archiving so many times
for maven-style projects? We are using Hudson version 1.211.

For example you can see that that every artifact below is archived 5
times during a single build. It seems that this has a huge impact on the
performance of our maven-style builds. The project converted from
maven-style to the free-style is built much, much faster.

...........
 [HUDSON] Recording test results
[INFO] [jar:jar]
[INFO] Building jar:
/usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
/a2/target/a2-1.0-SNAPSHOT.jar
[INFO] Preparing source:jar
[WARNING] Removing: jar from forked lifecycle, to prevent recursive
invocation.
[HUDSON] Archiving
/usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
/a2/pom.xml to
/usr/local/hudson/bin/../workspace/jobs/project2/modules/com.tvworks.tva
.test$a2/builds/2008-05-14_16-51-52/archive/com.tvworks.tva.test/a2/1.0-
SNAPSHOT/pom.xml
[HUDSON] Archiving
/usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
/a2/target/a2-1.0-SNAPSHOT.jar to
/usr/local/hudson/bin/../workspace/jobs/project2/modules/com.tvworks.tva
.test$a2/builds/2008-05-14_16-51-52/archive/com.tvworks.tva.test/a2/1.0-
SNAPSHOT/a2-1.0-SNAPSHOT.jar
[INFO] [enforcer:enforce {execution: enforce-versions}]
[INFO] The cached results are still valid. Skipping the rule:
org.apache.maven.plugins.enforcer.RequireMavenVersion
[INFO] The cached results are still valid. Skipping the rule:
org.apache.maven.plugins.enforcer.RequireJavaVersion
[HUDSON] Archiving
/usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
/a2/pom.xml to
/usr/local/hudson/bin/../workspace/jobs/project2/modules/com.tvworks.tva
.test$a2/builds/2008-05-14_16-51-52/archive/com.tvworks.tva.test/a2/1.0-
SNAPSHOT/pom.xml
[HUDSON] Archiving
/usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
/a2/target/a2-1.0-SNAPSHOT.jar to
/usr/local/hudson/bin/../workspace/jobs/project2/modules/com.tvworks.tva
.test$a2/builds/2008-05-14_16-51-52/archive/com.tvworks.tva.test/a2/1.0-
SNAPSHOT/a2-1.0-SNAPSHOT.jar
[INFO] [source:jar {execution: package-sources}]
[INFO] Building jar:
/usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
/a2/target/a2-1.0-SNAPSHOT-sources.jar
[INFO] Preparing source:test-jar
[WARNING] Removing: jar from forked lifecycle, to prevent recursive
invocation.
[WARNING] Removing: test-jar from forked lifecycle, to prevent recursive
invocation.
[HUDSON] Archiving
/usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
/a2/pom.xml to
/usr/local/hudson/bin/../workspace/jobs/project2/modules/com.tvworks.tva
.test$a2/builds/2008-05-14_16-51-52/archive/com.tvworks.tva.test/a2/1.0-
SNAPSHOT/pom.xml
[HUDSON] Archiving
/usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
/a2/target/a2-1.0-SNAPSHOT.jar to
/usr/local/hudson/bin/../workspace/jobs/project2/modules/com.tvworks.tva
.test$a2/builds/2008-05-14_16-51-52/archive/com.tvworks.tva.test/a2/1.0-
SNAPSHOT/a2-1.0-SNAPSHOT.jar
[HUDSON] Archiving
/usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
/a2/target/a2-1.0-SNAPSHOT-sources.jar to
/usr/local/hudson/bin/../workspace/jobs/project2/modules/com.tvworks.tva
.test$a2/builds/2008-05-14_16-51-52/archive/com.tvworks.tva.test/a2/1.0-
SNAPSHOT/a2-1.0-SNAPSHOT-sources.jar
[INFO] [enforcer:enforce {execution: enforce-versions}]
[INFO] The cached results are still valid. Skipping the rule:
org.apache.maven.plugins.enforcer.RequireMavenVersion
[INFO] The cached results are still valid. Skipping the rule:
org.apache.maven.plugins.enforcer.RequireJavaVersion
[HUDSON] Archiving
/usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
/a2/pom.xml to
/usr/local/hudson/bin/../workspace/jobs/project2/modules/com.tvworks.tva
.test$a2/builds/2008-05-14_16-51-52/archive/com.tvworks.tva.test/a2/1.0-
SNAPSHOT/pom.xml
[HUDSON] Archiving
/usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
/a2/target/a2-1.0-SNAPSHOT.jar to
/usr/local/hudson/bin/../workspace/jobs/project2/modules/com.tvworks.tva
.test$a2/builds/2008-05-14_16-51-52/archive/com.tvworks.tva.test/a2/1.0-
SNAPSHOT/a2-1.0-SNAPSHOT.jar
[HUDSON] Archiving
/usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
/a2/target/a2-1.0-SNAPSHOT-sources.jar to
/usr/local/hudson/bin/../workspace/jobs/project2/modules/com.tvworks.tva
.test$a2/builds/2008-05-14_16-51-52/archive/com.tvworks.tva.test/a2/1.0-
SNAPSHOT/a2-1.0-SNAPSHOT-sources.jar
[INFO] [source:test-jar {execution: package-sources}]
[INFO] Building jar:
/usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
/a2/target/a2-1.0-SNAPSHOT-test-sources.jar
[INFO] [install:install]
[INFO] Installing
/usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
/a2/target/a2-1.0-SNAPSHOT.jar to
/home/hudson/.m2/repository/com/tvworks/tva/test/a2/1.0-SNAPSHOT/a2-1.0-
SNAPSHOT.jar
[INFO] Installing
/usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
/a2/target/a2-1.0-SNAPSHOT-sources.jar to
/home/hudson/.m2/repository/com/tvworks/tva/test/a2/1.0-SNAPSHOT/a2-1.0-
SNAPSHOT-sources.jar
[INFO] Installing
/usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
/a2/target/a2-1.0-SNAPSHOT-test-sources.jar to
/home/hudson/.m2/repository/com/tvworks/tva/test/a2/1.0-SNAPSHOT/a2-1.0-
SNAPSHOT-test-sources.jar
[HUDSON] Archiving
/usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
/a2/pom.xml to
/usr/local/hudson/bin/../workspace/jobs/project2/modules/com.tvworks.tva
.test$a2/builds/2008-05-14_16-51-52/archive/com.tvworks.tva.test/a2/1.0-
SNAPSHOT/pom.xml
[HUDSON] Archiving
/usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
/a2/target/a2-1.0-SNAPSHOT.jar to
/usr/local/hudson/bin/../workspace/jobs/project2/modules/com.tvworks.tva
.test$a2/builds/2008-05-14_16-51-52/archive/com.tvworks.tva.test/a2/1.0-
SNAPSHOT/a2-1.0-SNAPSHOT.jar
[HUDSON] Archiving
/usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
/a2/target/a2-1.0-SNAPSHOT-sources.jar to
/usr/local/hudson/bin/../workspace/jobs/project2/modules/com.tvworks.tva
.test$a2/builds/2008-05-14_16-51-52/archive/com.tvworks.tva.test/a2/1.0-
SNAPSHOT/a2-1.0-SNAPSHOT-sources.jar
[HUDSON] Archiving
/usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
/a2/target/a2-1.0-SNAPSHOT-test-sources.jar to
/usr/local/hudson/bin/../workspace/jobs/project2/modules/com.tvworks.tva
.test$a2/builds/2008-05-14_16-51-52/archive/com.tvworks.tva.test/a2/1.0-
SNAPSHOT/a2-1.0-SNAPSHOT-test-sources.jar
[INFO]
[INFO]
.......

Thanks,
Seva


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Multiple archiving for maven-style projects

Kohsuke Kawaguchi
Administrator

Any chance you can create a test case for me?

What goal do you execute?

Seva Popov wrote:

> Hi,
>
> Could somebody explain why Hudson executes the archiving so many times
> for maven-style projects? We are using Hudson version 1.211.
>
> For example you can see that that every artifact below is archived 5
> times during a single build. It seems that this has a huge impact on the
> performance of our maven-style builds. The project converted from
> maven-style to the free-style is built much, much faster.
>
> ..........
>  [HUDSON] Recording test results
> [INFO] [jar:jar]
> [INFO] Building jar:
> /usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
> /a2/target/a2-1.0-SNAPSHOT.jar
> [INFO] Preparing source:jar
> [WARNING] Removing: jar from forked lifecycle, to prevent recursive
> invocation.
> [HUDSON] Archiving
> /usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
> /a2/pom.xml to
> /usr/local/hudson/bin/../workspace/jobs/project2/modules/com.tvworks.tva
> .test$a2/builds/2008-05-14_16-51-52/archive/com.tvworks.tva.test/a2/1.0-
> SNAPSHOT/pom.xml
> [HUDSON] Archiving
> /usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
> /a2/target/a2-1.0-SNAPSHOT.jar to
> /usr/local/hudson/bin/../workspace/jobs/project2/modules/com.tvworks.tva
> .test$a2/builds/2008-05-14_16-51-52/archive/com.tvworks.tva.test/a2/1.0-
> SNAPSHOT/a2-1.0-SNAPSHOT.jar
> [INFO] [enforcer:enforce {execution: enforce-versions}]
> [INFO] The cached results are still valid. Skipping the rule:
> org.apache.maven.plugins.enforcer.RequireMavenVersion
> [INFO] The cached results are still valid. Skipping the rule:
> org.apache.maven.plugins.enforcer.RequireJavaVersion
> [HUDSON] Archiving
> /usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
> /a2/pom.xml to
> /usr/local/hudson/bin/../workspace/jobs/project2/modules/com.tvworks.tva
> .test$a2/builds/2008-05-14_16-51-52/archive/com.tvworks.tva.test/a2/1.0-
> SNAPSHOT/pom.xml
> [HUDSON] Archiving
> /usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
> /a2/target/a2-1.0-SNAPSHOT.jar to
> /usr/local/hudson/bin/../workspace/jobs/project2/modules/com.tvworks.tva
> .test$a2/builds/2008-05-14_16-51-52/archive/com.tvworks.tva.test/a2/1.0-
> SNAPSHOT/a2-1.0-SNAPSHOT.jar
> [INFO] [source:jar {execution: package-sources}]
> [INFO] Building jar:
> /usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
> /a2/target/a2-1.0-SNAPSHOT-sources.jar
> [INFO] Preparing source:test-jar
> [WARNING] Removing: jar from forked lifecycle, to prevent recursive
> invocation.
> [WARNING] Removing: test-jar from forked lifecycle, to prevent recursive
> invocation.
> [HUDSON] Archiving
> /usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
> /a2/pom.xml to
> /usr/local/hudson/bin/../workspace/jobs/project2/modules/com.tvworks.tva
> .test$a2/builds/2008-05-14_16-51-52/archive/com.tvworks.tva.test/a2/1.0-
> SNAPSHOT/pom.xml
> [HUDSON] Archiving
> /usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
> /a2/target/a2-1.0-SNAPSHOT.jar to
> /usr/local/hudson/bin/../workspace/jobs/project2/modules/com.tvworks.tva
> .test$a2/builds/2008-05-14_16-51-52/archive/com.tvworks.tva.test/a2/1.0-
> SNAPSHOT/a2-1.0-SNAPSHOT.jar
> [HUDSON] Archiving
> /usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
> /a2/target/a2-1.0-SNAPSHOT-sources.jar to
> /usr/local/hudson/bin/../workspace/jobs/project2/modules/com.tvworks.tva
> .test$a2/builds/2008-05-14_16-51-52/archive/com.tvworks.tva.test/a2/1.0-
> SNAPSHOT/a2-1.0-SNAPSHOT-sources.jar
> [INFO] [enforcer:enforce {execution: enforce-versions}]
> [INFO] The cached results are still valid. Skipping the rule:
> org.apache.maven.plugins.enforcer.RequireMavenVersion
> [INFO] The cached results are still valid. Skipping the rule:
> org.apache.maven.plugins.enforcer.RequireJavaVersion
> [HUDSON] Archiving
> /usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
> /a2/pom.xml to
> /usr/local/hudson/bin/../workspace/jobs/project2/modules/com.tvworks.tva
> .test$a2/builds/2008-05-14_16-51-52/archive/com.tvworks.tva.test/a2/1.0-
> SNAPSHOT/pom.xml
> [HUDSON] Archiving
> /usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
> /a2/target/a2-1.0-SNAPSHOT.jar to
> /usr/local/hudson/bin/../workspace/jobs/project2/modules/com.tvworks.tva
> .test$a2/builds/2008-05-14_16-51-52/archive/com.tvworks.tva.test/a2/1.0-
> SNAPSHOT/a2-1.0-SNAPSHOT.jar
> [HUDSON] Archiving
> /usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
> /a2/target/a2-1.0-SNAPSHOT-sources.jar to
> /usr/local/hudson/bin/../workspace/jobs/project2/modules/com.tvworks.tva
> .test$a2/builds/2008-05-14_16-51-52/archive/com.tvworks.tva.test/a2/1.0-
> SNAPSHOT/a2-1.0-SNAPSHOT-sources.jar
> [INFO] [source:test-jar {execution: package-sources}]
> [INFO] Building jar:
> /usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
> /a2/target/a2-1.0-SNAPSHOT-test-sources.jar
> [INFO] [install:install]
> [INFO] Installing
> /usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
> /a2/target/a2-1.0-SNAPSHOT.jar to
> /home/hudson/.m2/repository/com/tvworks/tva/test/a2/1.0-SNAPSHOT/a2-1.0-
> SNAPSHOT.jar
> [INFO] Installing
> /usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
> /a2/target/a2-1.0-SNAPSHOT-sources.jar to
> /home/hudson/.m2/repository/com/tvworks/tva/test/a2/1.0-SNAPSHOT/a2-1.0-
> SNAPSHOT-sources.jar
> [INFO] Installing
> /usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
> /a2/target/a2-1.0-SNAPSHOT-test-sources.jar to
> /home/hudson/.m2/repository/com/tvworks/tva/test/a2/1.0-SNAPSHOT/a2-1.0-
> SNAPSHOT-test-sources.jar
> [HUDSON] Archiving
> /usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
> /a2/pom.xml to
> /usr/local/hudson/bin/../workspace/jobs/project2/modules/com.tvworks.tva
> .test$a2/builds/2008-05-14_16-51-52/archive/com.tvworks.tva.test/a2/1.0-
> SNAPSHOT/pom.xml
> [HUDSON] Archiving
> /usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
> /a2/target/a2-1.0-SNAPSHOT.jar to
> /usr/local/hudson/bin/../workspace/jobs/project2/modules/com.tvworks.tva
> .test$a2/builds/2008-05-14_16-51-52/archive/com.tvworks.tva.test/a2/1.0-
> SNAPSHOT/a2-1.0-SNAPSHOT.jar
> [HUDSON] Archiving
> /usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
> /a2/target/a2-1.0-SNAPSHOT-sources.jar to
> /usr/local/hudson/bin/../workspace/jobs/project2/modules/com.tvworks.tva
> .test$a2/builds/2008-05-14_16-51-52/archive/com.tvworks.tva.test/a2/1.0-
> SNAPSHOT/a2-1.0-SNAPSHOT-sources.jar
> [HUDSON] Archiving
> /usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
> /a2/target/a2-1.0-SNAPSHOT-test-sources.jar to
> /usr/local/hudson/bin/../workspace/jobs/project2/modules/com.tvworks.tva
> .test$a2/builds/2008-05-14_16-51-52/archive/com.tvworks.tva.test/a2/1.0-
> SNAPSHOT/a2-1.0-SNAPSHOT-test-sources.jar
> [INFO]
> [INFO]
> ......
>
> Thanks,
> Seva
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>

--
Kohsuke Kawaguchi
Sun Microsystems                   [hidden email]

smime.p7s (4K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

RE: Re: Multiple archiving for maven-style projects

Seva Popov
Kohsuke,

The issue happens when you execute either the 'mvn clean install' or
'mvn clean deploy' goals.

While preparing the test case for you I've been able to narrow down the
exact configuration in the pom that causes this issue.

Below I am pasting the email that I've sent to the developers of my
company:

The issue with multiple archiving should be fixed now. We've also fixed
the corresponding issue of multiple invocations of the enforcer plugin
(i.e. the enforcer plugin was executed many times for every module).
You should be able to see the considerable decrease in the build times
for the maven-style projects.

The issue was due to the combination of the following two factors: our
usage of maven-enforcer plugin and maven-source plugin plus an
identified impact that this had to do on Hudson (I am in contact with
Hudson lead on this issue).

Below are the relevant plugins that are specified in the tva pom.xml
file. We were using the maven-enforcer plugin without a lifecycle phase
binding so the default one was used which is 'validate'.  Now, the
maven-source plugin is forking additional lifecycle, namely
'generate-sources' before it executes any goal. This resulted in the
maven-enforcer plugin executed many times (and it looks like Hudson
archiving is also bounded to the lifecycles). So, the solution was to
change the binding of the maven-enforcer plugin from the default one to
one of the phases that goes after 'generate-sources' phase in order to
prevent multiple invocations. The next phase after 'generate-sources'
phase is 'process-sources' phase and so it was chosen for the
maven-enforcer plugin.

    <plugins>

      <plugin>

        <groupId>org.apache.maven.plugins</groupId>

        <artifactId>maven-enforcer-plugin</artifactId>

        <executions>

          <execution>

            <id>enforce-versions</id>

            <!-- should be validate phase but moving after generate
sources phase to prevent execution by the maven-source-plugin -->

            <phase>process-sources</phase>

            <goals>

              <goal>enforce</goal>

            </goals>

          </execution>

        </executions>

      </plugin>

      <plugin>

        <groupId>org.apache.maven.plugins</groupId>

        <artifactId>maven-source-plugin</artifactId>

        <executions>

          <execution>

            <id>package-sources</id>

            <phase>package</phase>

            <goals>

              <goal>jar</goal>

              <goal>test-jar</goal>

            </goals>

          </execution>

        </executions>

      </plugin>

First results are very good. For example the MSS-3.3 project that used
to require 42 minutes for building was built in just 23 minutes.

Thanks,
Seva

-----Original Message-----
From: Kohsuke Kawaguchi [mailto:[hidden email]]
Sent: Monday, May 19, 2008 10:55 AM
To: [hidden email]
Subject: Re: Multiple archiving for maven-style projects


Any chance you can create a test case for me?

What goal do you execute?

Seva Popov wrote:
> Hi,
>
> Could somebody explain why Hudson executes the archiving so many times
> for maven-style projects? We are using Hudson version 1.211.
>
> For example you can see that that every artifact below is archived 5
> times during a single build. It seems that this has a huge impact on
the
> performance of our maven-style builds. The project converted from
> maven-style to the free-style is built much, much faster.
>
> ..........
>  [HUDSON] Recording test results
> [INFO] [jar:jar]
> [INFO] Building jar:
>
/usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
> /a2/target/a2-1.0-SNAPSHOT.jar
> [INFO] Preparing source:jar
> [WARNING] Removing: jar from forked lifecycle, to prevent recursive
> invocation.
> [HUDSON] Archiving
>
/usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
> /a2/pom.xml to
>
/usr/local/hudson/bin/../workspace/jobs/project2/modules/com.tvworks.tva
>
.test$a2/builds/2008-05-14_16-51-52/archive/com.tvworks.tva.test/a2/1.0-
> SNAPSHOT/pom.xml
> [HUDSON] Archiving
>
/usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
> /a2/target/a2-1.0-SNAPSHOT.jar to
>
/usr/local/hudson/bin/../workspace/jobs/project2/modules/com.tvworks.tva
>
.test$a2/builds/2008-05-14_16-51-52/archive/com.tvworks.tva.test/a2/1.0-
> SNAPSHOT/a2-1.0-SNAPSHOT.jar
> [INFO] [enforcer:enforce {execution: enforce-versions}]
> [INFO] The cached results are still valid. Skipping the rule:
> org.apache.maven.plugins.enforcer.RequireMavenVersion
> [INFO] The cached results are still valid. Skipping the rule:
> org.apache.maven.plugins.enforcer.RequireJavaVersion
> [HUDSON] Archiving
>
/usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
> /a2/pom.xml to
>
/usr/local/hudson/bin/../workspace/jobs/project2/modules/com.tvworks.tva
>
.test$a2/builds/2008-05-14_16-51-52/archive/com.tvworks.tva.test/a2/1.0-
> SNAPSHOT/pom.xml
> [HUDSON] Archiving
>
/usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
> /a2/target/a2-1.0-SNAPSHOT.jar to
>
/usr/local/hudson/bin/../workspace/jobs/project2/modules/com.tvworks.tva
>
.test$a2/builds/2008-05-14_16-51-52/archive/com.tvworks.tva.test/a2/1.0-
> SNAPSHOT/a2-1.0-SNAPSHOT.jar
> [INFO] [source:jar {execution: package-sources}]
> [INFO] Building jar:
>
/usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
> /a2/target/a2-1.0-SNAPSHOT-sources.jar
> [INFO] Preparing source:test-jar
> [WARNING] Removing: jar from forked lifecycle, to prevent recursive
> invocation.
> [WARNING] Removing: test-jar from forked lifecycle, to prevent
recursive
> invocation.
> [HUDSON] Archiving
>
/usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
> /a2/pom.xml to
>
/usr/local/hudson/bin/../workspace/jobs/project2/modules/com.tvworks.tva
>
.test$a2/builds/2008-05-14_16-51-52/archive/com.tvworks.tva.test/a2/1.0-
> SNAPSHOT/pom.xml
> [HUDSON] Archiving
>
/usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
> /a2/target/a2-1.0-SNAPSHOT.jar to
>
/usr/local/hudson/bin/../workspace/jobs/project2/modules/com.tvworks.tva
>
.test$a2/builds/2008-05-14_16-51-52/archive/com.tvworks.tva.test/a2/1.0-
> SNAPSHOT/a2-1.0-SNAPSHOT.jar
> [HUDSON] Archiving
>
/usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
> /a2/target/a2-1.0-SNAPSHOT-sources.jar to
>
/usr/local/hudson/bin/../workspace/jobs/project2/modules/com.tvworks.tva
>
.test$a2/builds/2008-05-14_16-51-52/archive/com.tvworks.tva.test/a2/1.0-
> SNAPSHOT/a2-1.0-SNAPSHOT-sources.jar
> [INFO] [enforcer:enforce {execution: enforce-versions}]
> [INFO] The cached results are still valid. Skipping the rule:
> org.apache.maven.plugins.enforcer.RequireMavenVersion
> [INFO] The cached results are still valid. Skipping the rule:
> org.apache.maven.plugins.enforcer.RequireJavaVersion
> [HUDSON] Archiving
>
/usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
> /a2/pom.xml to
>
/usr/local/hudson/bin/../workspace/jobs/project2/modules/com.tvworks.tva
>
.test$a2/builds/2008-05-14_16-51-52/archive/com.tvworks.tva.test/a2/1.0-
> SNAPSHOT/pom.xml
> [HUDSON] Archiving
>
/usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
> /a2/target/a2-1.0-SNAPSHOT.jar to
>
/usr/local/hudson/bin/../workspace/jobs/project2/modules/com.tvworks.tva
>
.test$a2/builds/2008-05-14_16-51-52/archive/com.tvworks.tva.test/a2/1.0-
> SNAPSHOT/a2-1.0-SNAPSHOT.jar
> [HUDSON] Archiving
>
/usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
> /a2/target/a2-1.0-SNAPSHOT-sources.jar to
>
/usr/local/hudson/bin/../workspace/jobs/project2/modules/com.tvworks.tva
>
.test$a2/builds/2008-05-14_16-51-52/archive/com.tvworks.tva.test/a2/1.0-
> SNAPSHOT/a2-1.0-SNAPSHOT-sources.jar
> [INFO] [source:test-jar {execution: package-sources}]
> [INFO] Building jar:
>
/usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
> /a2/target/a2-1.0-SNAPSHOT-test-sources.jar
> [INFO] [install:install]
> [INFO] Installing
>
/usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
> /a2/target/a2-1.0-SNAPSHOT.jar to
>
/home/hudson/.m2/repository/com/tvworks/tva/test/a2/1.0-SNAPSHOT/a2-1.0-
> SNAPSHOT.jar
> [INFO] Installing
>
/usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
> /a2/target/a2-1.0-SNAPSHOT-sources.jar to
>
/home/hudson/.m2/repository/com/tvworks/tva/test/a2/1.0-SNAPSHOT/a2-1.0-
> SNAPSHOT-sources.jar
> [INFO] Installing
>
/usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
> /a2/target/a2-1.0-SNAPSHOT-test-sources.jar to
>
/home/hudson/.m2/repository/com/tvworks/tva/test/a2/1.0-SNAPSHOT/a2-1.0-
> SNAPSHOT-test-sources.jar
> [HUDSON] Archiving
>
/usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
> /a2/pom.xml to
>
/usr/local/hudson/bin/../workspace/jobs/project2/modules/com.tvworks.tva
>
.test$a2/builds/2008-05-14_16-51-52/archive/com.tvworks.tva.test/a2/1.0-
> SNAPSHOT/pom.xml
> [HUDSON] Archiving
>
/usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
> /a2/target/a2-1.0-SNAPSHOT.jar to
>
/usr/local/hudson/bin/../workspace/jobs/project2/modules/com.tvworks.tva
>
.test$a2/builds/2008-05-14_16-51-52/archive/com.tvworks.tva.test/a2/1.0-
> SNAPSHOT/a2-1.0-SNAPSHOT.jar
> [HUDSON] Archiving
>
/usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
> /a2/target/a2-1.0-SNAPSHOT-sources.jar to
>
/usr/local/hudson/bin/../workspace/jobs/project2/modules/com.tvworks.tva
>
.test$a2/builds/2008-05-14_16-51-52/archive/com.tvworks.tva.test/a2/1.0-
> SNAPSHOT/a2-1.0-SNAPSHOT-sources.jar
> [HUDSON] Archiving
>
/usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
> /a2/target/a2-1.0-SNAPSHOT-test-sources.jar to
>
/usr/local/hudson/bin/../workspace/jobs/project2/modules/com.tvworks.tva
>
.test$a2/builds/2008-05-14_16-51-52/archive/com.tvworks.tva.test/a2/1.0-

> SNAPSHOT/a2-1.0-SNAPSHOT-test-sources.jar
> [INFO]
> [INFO]
> ......
>
> Thanks,
> Seva
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>


--
Kohsuke Kawaguchi
Sun Microsystems                   [hidden email]

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Multiple archiving for maven-style projects

Kohsuke Kawaguchi
Administrator

Thank you for the analysis. I'd still like to fix the bug in Hudson ---
if I understand it correctly, the issue is Hudson doesn't understand
when Maven forks another life cycle (to be honest, *I* don't understand
another life cycle phase!)

So any test case would be still greatly appreciated.

Seva Popov wrote:

> Kohsuke,
>
> The issue happens when you execute either the 'mvn clean install' or
> 'mvn clean deploy' goals.
>
> While preparing the test case for you I've been able to narrow down the
> exact configuration in the pom that causes this issue.
>
> Below I am pasting the email that I've sent to the developers of my
> company:
>
> The issue with multiple archiving should be fixed now. We've also fixed
> the corresponding issue of multiple invocations of the enforcer plugin
> (i.e. the enforcer plugin was executed many times for every module).
> You should be able to see the considerable decrease in the build times
> for the maven-style projects.
>
> The issue was due to the combination of the following two factors: our
> usage of maven-enforcer plugin and maven-source plugin plus an
> identified impact that this had to do on Hudson (I am in contact with
> Hudson lead on this issue).
>
> Below are the relevant plugins that are specified in the tva pom.xml
> file. We were using the maven-enforcer plugin without a lifecycle phase
> binding so the default one was used which is 'validate'.  Now, the
> maven-source plugin is forking additional lifecycle, namely
> 'generate-sources' before it executes any goal. This resulted in the
> maven-enforcer plugin executed many times (and it looks like Hudson
> archiving is also bounded to the lifecycles). So, the solution was to
> change the binding of the maven-enforcer plugin from the default one to
> one of the phases that goes after 'generate-sources' phase in order to
> prevent multiple invocations. The next phase after 'generate-sources'
> phase is 'process-sources' phase and so it was chosen for the
> maven-enforcer plugin.
>
>     <plugins>
>
>       <plugin>
>
>         <groupId>org.apache.maven.plugins</groupId>
>
>         <artifactId>maven-enforcer-plugin</artifactId>
>
>         <executions>
>
>           <execution>
>
>             <id>enforce-versions</id>
>
>             <!-- should be validate phase but moving after generate
> sources phase to prevent execution by the maven-source-plugin -->
>
>             <phase>process-sources</phase>
>
>             <goals>
>
>               <goal>enforce</goal>
>
>             </goals>
>
>           </execution>
>
>         </executions>
>
>       </plugin>
>
>       <plugin>
>
>         <groupId>org.apache.maven.plugins</groupId>
>
>         <artifactId>maven-source-plugin</artifactId>
>
>         <executions>
>
>           <execution>
>
>             <id>package-sources</id>
>
>             <phase>package</phase>
>
>             <goals>
>
>               <goal>jar</goal>
>
>               <goal>test-jar</goal>
>
>             </goals>
>
>           </execution>
>
>         </executions>
>
>       </plugin>
>
> First results are very good. For example the MSS-3.3 project that used
> to require 42 minutes for building was built in just 23 minutes.
>
> Thanks,
> Seva
>
> -----Original Message-----
> From: Kohsuke Kawaguchi [mailto:[hidden email]]
> Sent: Monday, May 19, 2008 10:55 AM
> To: [hidden email]
> Subject: Re: Multiple archiving for maven-style projects
>
>
> Any chance you can create a test case for me?
>
> What goal do you execute?
>
> Seva Popov wrote:
>> Hi,
>>
>> Could somebody explain why Hudson executes the archiving so many times
>> for maven-style projects? We are using Hudson version 1.211.
>>
>> For example you can see that that every artifact below is archived 5
>> times during a single build. It seems that this has a huge impact on
> the
>> performance of our maven-style builds. The project converted from
>> maven-style to the free-style is built much, much faster.
>>
>> ..........
>>  [HUDSON] Recording test results
>> [INFO] [jar:jar]
>> [INFO] Building jar:
>>
> /usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
>> /a2/target/a2-1.0-SNAPSHOT.jar
>> [INFO] Preparing source:jar
>> [WARNING] Removing: jar from forked lifecycle, to prevent recursive
>> invocation.
>> [HUDSON] Archiving
>>
> /usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
>> /a2/pom.xml to
>>
> /usr/local/hudson/bin/../workspace/jobs/project2/modules/com.tvworks.tva
>>
> .test$a2/builds/2008-05-14_16-51-52/archive/com.tvworks.tva.test/a2/1.0-
>> SNAPSHOT/pom.xml
>> [HUDSON] Archiving
>>
> /usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
>> /a2/target/a2-1.0-SNAPSHOT.jar to
>>
> /usr/local/hudson/bin/../workspace/jobs/project2/modules/com.tvworks.tva
>>
> .test$a2/builds/2008-05-14_16-51-52/archive/com.tvworks.tva.test/a2/1.0-
>> SNAPSHOT/a2-1.0-SNAPSHOT.jar
>> [INFO] [enforcer:enforce {execution: enforce-versions}]
>> [INFO] The cached results are still valid. Skipping the rule:
>> org.apache.maven.plugins.enforcer.RequireMavenVersion
>> [INFO] The cached results are still valid. Skipping the rule:
>> org.apache.maven.plugins.enforcer.RequireJavaVersion
>> [HUDSON] Archiving
>>
> /usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
>> /a2/pom.xml to
>>
> /usr/local/hudson/bin/../workspace/jobs/project2/modules/com.tvworks.tva
>>
> .test$a2/builds/2008-05-14_16-51-52/archive/com.tvworks.tva.test/a2/1.0-
>> SNAPSHOT/pom.xml
>> [HUDSON] Archiving
>>
> /usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
>> /a2/target/a2-1.0-SNAPSHOT.jar to
>>
> /usr/local/hudson/bin/../workspace/jobs/project2/modules/com.tvworks.tva
>>
> .test$a2/builds/2008-05-14_16-51-52/archive/com.tvworks.tva.test/a2/1.0-
>> SNAPSHOT/a2-1.0-SNAPSHOT.jar
>> [INFO] [source:jar {execution: package-sources}]
>> [INFO] Building jar:
>>
> /usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
>> /a2/target/a2-1.0-SNAPSHOT-sources.jar
>> [INFO] Preparing source:test-jar
>> [WARNING] Removing: jar from forked lifecycle, to prevent recursive
>> invocation.
>> [WARNING] Removing: test-jar from forked lifecycle, to prevent
> recursive
>> invocation.
>> [HUDSON] Archiving
>>
> /usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
>> /a2/pom.xml to
>>
> /usr/local/hudson/bin/../workspace/jobs/project2/modules/com.tvworks.tva
>>
> .test$a2/builds/2008-05-14_16-51-52/archive/com.tvworks.tva.test/a2/1.0-
>> SNAPSHOT/pom.xml
>> [HUDSON] Archiving
>>
> /usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
>> /a2/target/a2-1.0-SNAPSHOT.jar to
>>
> /usr/local/hudson/bin/../workspace/jobs/project2/modules/com.tvworks.tva
>>
> .test$a2/builds/2008-05-14_16-51-52/archive/com.tvworks.tva.test/a2/1.0-
>> SNAPSHOT/a2-1.0-SNAPSHOT.jar
>> [HUDSON] Archiving
>>
> /usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
>> /a2/target/a2-1.0-SNAPSHOT-sources.jar to
>>
> /usr/local/hudson/bin/../workspace/jobs/project2/modules/com.tvworks.tva
>>
> .test$a2/builds/2008-05-14_16-51-52/archive/com.tvworks.tva.test/a2/1.0-
>> SNAPSHOT/a2-1.0-SNAPSHOT-sources.jar
>> [INFO] [enforcer:enforce {execution: enforce-versions}]
>> [INFO] The cached results are still valid. Skipping the rule:
>> org.apache.maven.plugins.enforcer.RequireMavenVersion
>> [INFO] The cached results are still valid. Skipping the rule:
>> org.apache.maven.plugins.enforcer.RequireJavaVersion
>> [HUDSON] Archiving
>>
> /usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
>> /a2/pom.xml to
>>
> /usr/local/hudson/bin/../workspace/jobs/project2/modules/com.tvworks.tva
>>
> .test$a2/builds/2008-05-14_16-51-52/archive/com.tvworks.tva.test/a2/1.0-
>> SNAPSHOT/pom.xml
>> [HUDSON] Archiving
>>
> /usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
>> /a2/target/a2-1.0-SNAPSHOT.jar to
>>
> /usr/local/hudson/bin/../workspace/jobs/project2/modules/com.tvworks.tva
>>
> .test$a2/builds/2008-05-14_16-51-52/archive/com.tvworks.tva.test/a2/1.0-
>> SNAPSHOT/a2-1.0-SNAPSHOT.jar
>> [HUDSON] Archiving
>>
> /usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
>> /a2/target/a2-1.0-SNAPSHOT-sources.jar to
>>
> /usr/local/hudson/bin/../workspace/jobs/project2/modules/com.tvworks.tva
>>
> .test$a2/builds/2008-05-14_16-51-52/archive/com.tvworks.tva.test/a2/1.0-
>> SNAPSHOT/a2-1.0-SNAPSHOT-sources.jar
>> [INFO] [source:test-jar {execution: package-sources}]
>> [INFO] Building jar:
>>
> /usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
>> /a2/target/a2-1.0-SNAPSHOT-test-sources.jar
>> [INFO] [install:install]
>> [INFO] Installing
>>
> /usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
>> /a2/target/a2-1.0-SNAPSHOT.jar to
>>
> /home/hudson/.m2/repository/com/tvworks/tva/test/a2/1.0-SNAPSHOT/a2-1.0-
>> SNAPSHOT.jar
>> [INFO] Installing
>>
> /usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
>> /a2/target/a2-1.0-SNAPSHOT-sources.jar to
>>
> /home/hudson/.m2/repository/com/tvworks/tva/test/a2/1.0-SNAPSHOT/a2-1.0-
>> SNAPSHOT-sources.jar
>> [INFO] Installing
>>
> /usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
>> /a2/target/a2-1.0-SNAPSHOT-test-sources.jar to
>>
> /home/hudson/.m2/repository/com/tvworks/tva/test/a2/1.0-SNAPSHOT/a2-1.0-
>> SNAPSHOT-test-sources.jar
>> [HUDSON] Archiving
>>
> /usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
>> /a2/pom.xml to
>>
> /usr/local/hudson/bin/../workspace/jobs/project2/modules/com.tvworks.tva
>>
> .test$a2/builds/2008-05-14_16-51-52/archive/com.tvworks.tva.test/a2/1.0-
>> SNAPSHOT/pom.xml
>> [HUDSON] Archiving
>>
> /usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
>> /a2/target/a2-1.0-SNAPSHOT.jar to
>>
> /usr/local/hudson/bin/../workspace/jobs/project2/modules/com.tvworks.tva
>>
> .test$a2/builds/2008-05-14_16-51-52/archive/com.tvworks.tva.test/a2/1.0-
>> SNAPSHOT/a2-1.0-SNAPSHOT.jar
>> [HUDSON] Archiving
>>
> /usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
>> /a2/target/a2-1.0-SNAPSHOT-sources.jar to
>>
> /usr/local/hudson/bin/../workspace/jobs/project2/modules/com.tvworks.tva
>>
> .test$a2/builds/2008-05-14_16-51-52/archive/com.tvworks.tva.test/a2/1.0-
>> SNAPSHOT/a2-1.0-SNAPSHOT-sources.jar
>> [HUDSON] Archiving
>>
> /usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2
>> /a2/target/a2-1.0-SNAPSHOT-test-sources.jar to
>>
> /usr/local/hudson/bin/../workspace/jobs/project2/modules/com.tvworks.tva
>>
> .test$a2/builds/2008-05-14_16-51-52/archive/com.tvworks.tva.test/a2/1.0-
>> SNAPSHOT/a2-1.0-SNAPSHOT-test-sources.jar
>> [INFO]
>> [INFO]
>> ......
>>
>> Thanks,
>> Seva
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>>
>
>

--
Kohsuke Kawaguchi
Sun Microsystems                   [hidden email]

smime.p7s (4K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

RE: Re: Multiple archiving for maven-style projects

Seva Popov

Kohsuke,

 

My understanding is that it is the *combination* of the maven-enforcer-plugin and maven-source-plugin specified in the pom that results in multiple archiving. If you use the maven-source-plugin alone (which is actually the one that forks another maven lifecycle) than the multiple archiving does not happen. So, I am not really saying that forking of another maven lifecycle is the direct reason of this issue.

 

Test case.

 

Building the following pom.xml file:

 

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">

  <modelVersion>4.0.0</modelVersion>

  <groupId>com.tvworks.tva.test</groupId>

  <artifactId>tva</artifactId>

  <name>tva</name>

  <version>1.0-SNAPSHOT</version>

  <packaging>pom</packaging>

  <build>

    <pluginManagement>

      <plugins>

        <plugin>

          <groupId>org.apache.maven.plugins</groupId>

          <artifactId>maven-enforcer-plugin</artifactId>

          <version>1.0-alpha-3</version>

          <configuration>

            <rules>

              <requireMavenVersion>

                <version>[2.0.7, 2.0.9]</version>

                <message>Maven version: 2.0.7 or 2.0.8 or 2.0.9 is required</message>

              </requireMavenVersion>

              <requireJavaVersion>

                <version>[1.5.0-12,)</version>

                <message>Java version: 1.5.0_12 is required</message>

              </requireJavaVersion>

            </rules>

          </configuration>

        </plugin>

        <plugin>

          <groupId>org.apache.maven.plugins</groupId>

          <artifactId>maven-source-plugin</artifactId>

          <version>2.0.4</version>

        </plugin>

      </plugins>

    </pluginManagement>

    <plugins>

      <plugin>

        <groupId>org.apache.maven.plugins</groupId>

        <artifactId>maven-enforcer-plugin</artifactId>

        <executions>

          <execution>

            <id>enforce-versions</id>

            <goals>

              <goal>enforce</goal>

            </goals>

          </execution>

        </executions>

      </plugin>

      <plugin>

        <groupId>org.apache.maven.plugins</groupId>

        <artifactId>maven-source-plugin</artifactId>

        <executions>

          <execution>

            <id>package-sources</id>

            <phase>package</phase>

            <goals>

              <goal>jar</goal>

              <goal>test-jar</goal>

            </goals>

          </execution>

        </executions>

      </plugin>

    </plugins>

  </build>

</project>

 

Produces the output with multiple archiving:

 

started
Updating http://svn.tva.tvworks.com/svn/tvworks/trunk/test/poms
U         pom.xml
At revision 50740
Parsing POMs
[poms] $ /usr/local/software/jdk1.5.0_12/bin/java -cp /tmp/hudson/winstone/hudson.war/WEB-INF/lib/maven-agent-1.211.jar:/usr/local/software/apache-maven-2.0.9/boot/classworlds-1.1.jar hudson.maven.agent.Main /usr/local/software/apache-maven-2.0.9 /tmp/hudson/winstone/hudson.war/WEB-INF/lib/remoting-1.211.jar /tmp/hudson/winstone/hudson.war/WEB-INF/lib/maven-interceptor-1.211.jar
channel started
[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Building tva
[INFO]    task-segment: [clean, install]
[INFO] ------------------------------------------------------------------------
[INFO] [clean:clean]
[INFO] [enforcer:enforce {execution: enforce-versions}]
[INFO] [site:attach-descriptor]
[INFO] Preparing source:jar
[WARNING] Removing: jar from forked lifecycle, to prevent recursive invocation.
[HUDSON] Archiving /usr/local/hudson/bin/../workspace/jobs/poms/workspace/poms/pom.xml to /usr/local/hudson/bin/../workspace/jobs/poms/modules/com.tvworks.tva.test$tva/builds/2008-05-21_11-05-58/archive/com.tvworks.tva.test/tva/1.0-SNAPSHOT/pom.xml
[INFO] [enforcer:enforce {execution: enforce-versions}]
[HUDSON] Archiving /usr/local/hudson/bin/../workspace/jobs/poms/workspace/poms/pom.xml to /usr/local/hudson/bin/../workspace/jobs/poms/modules/com.tvworks.tva.test$tva/builds/2008-05-21_11-05-58/archive/com.tvworks.tva.test/tva/1.0-SNAPSHOT/pom.xml
[INFO] [source:jar {execution: package-sources}]
[INFO] Preparing source:test-jar
[WARNING] Removing: jar from forked lifecycle, to prevent recursive invocation.
[WARNING] Removing: test-jar from forked lifecycle, to prevent recursive invocation.
[HUDSON] Archiving /usr/local/hudson/bin/../workspace/jobs/poms/workspace/poms/pom.xml to /usr/local/hudson/bin/../workspace/jobs/poms/modules/com.tvworks.tva.test$tva/builds/2008-05-21_11-05-58/archive/com.tvworks.tva.test/tva/1.0-SNAPSHOT/pom.xml
[INFO] [enforcer:enforce {execution: enforce-versions}]
[HUDSON] Archiving /usr/local/hudson/bin/../workspace/jobs/poms/workspace/poms/pom.xml to /usr/local/hudson/bin/../workspace/jobs/poms/modules/com.tvworks.tva.test$tva/builds/2008-05-21_11-05-58/archive/com.tvworks.tva.test/tva/1.0-SNAPSHOT/pom.xml
[INFO] [source:test-jar {execution: package-sources}]
[INFO] [install:install]
[INFO] Installing /usr/local/hudson/bin/../workspace/jobs/poms/workspace/poms/pom.xml to /home/hudson/.m2/repository/com/tvworks/tva/test/tva/1.0-SNAPSHOT/tva-1.0-SNAPSHOT.pom
[HUDSON] Archiving /usr/local/hudson/bin/../workspace/jobs/poms/workspace/poms/pom.xml to /usr/local/hudson/bin/../workspace/jobs/poms/modules/com.tvworks.tva.test$tva/builds/2008-05-21_11-05-58/archive/com.tvworks.tva.test/tva/1.0-SNAPSHOT/pom.xml
[INFO ] Installing /usr/local/hudson/bin/../workspace/jobs/poms/modules/com.tvworks.tva.test$tva/builds/2008-05-21_11-05-58/archive/com.tvworks.tva.test/tva/1.0-SNAPSHOT/pom.xml to /home/hudson/.m2/repository/com/tvworks/tva/test/tva/1.0-SNAPSHOT/tva-1.0-SNAPSHOT.pom
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 26 seconds
[INFO] Finished at: Wed May 21 11:06:29 PDT 2008
[INFO] Final Memory: 10M/19M
[INFO] ------------------------------------------------------------------------
channel stopped
Triggering a new build of masterpom
finished: SUCCESS

 

 

 

Thanks,

Seva

 

-----Original Message-----
From: Kohsuke Kawaguchi [mailto:[hidden email]]
Sent: Tuesday, May 20, 2008 6:49 PM
To: [hidden email]
Subject: Re: Multiple archiving for maven-style projects

 

 

Thank you for the analysis. I'd still like to fix the bug in Hudson ---

if I understand it correctly, the issue is Hudson doesn't understand

when Maven forks another life cycle (to be honest, *I* don't understand

another life cycle phase!)

 

So any test case would be still greatly appreciated.

 

Seva Popov wrote:

> Kohsuke,

>

> The issue happens when you execute either the 'mvn clean install' or

> 'mvn clean deploy' goals.

>

> While preparing the test case for you I've been able to narrow down the

> exact configuration in the pom that causes this issue.

>

> Below I am pasting the email that I've sent to the developers of my

> company:

>

> The issue with multiple archiving should be fixed now. We've also fixed

> the corresponding issue of multiple invocations of the enforcer plugin

> (i.e. the enforcer plugin was executed many times for every module).

> You should be able to see the considerable decrease in the build times

> for the maven-style projects.

>

> The issue was due to the combination of the following two factors: our

> usage of maven-enforcer plugin and maven-source plugin plus an

> identified impact that this had to do on Hudson (I am in contact with

> Hudson lead on this issue).

>

> Below are the relevant plugins that are specified in the tva pom.xml

> file. We were using the maven-enforcer plugin without a lifecycle phase

> binding so the default one was used which is 'validate'.  Now, the

> maven-source plugin is forking additional lifecycle, namely

> 'generate-sources' before it executes any goal. This resulted in the

> maven-enforcer plugin executed many times (and it looks like Hudson

> archiving is also bounded to the lifecycles). So, the solution was to

> change the binding of the maven-enforcer plugin from the default one to

> one of the phases that goes after 'generate-sources' phase in order to

> prevent multiple invocations. The next phase after 'generate-sources'

> phase is 'process-sources' phase and so it was chosen for the

> maven-enforcer plugin.

>

>     <plugins>

>

>       <plugin>

>

>         <groupId>org.apache.maven.plugins</groupId>

>

>         <artifactId>maven-enforcer-plugin</artifactId>

>

>         <executions>

>

>           <execution>

>

>             <id>enforce-versions</id>

>

>             <!-- should be validate phase but moving after generate

> sources phase to prevent execution by the maven-source-plugin -->

>

>             <phase>process-sources</phase>

>

>             <goals>

>

>               <goal>enforce</goal>

>

>             </goals>

>

>           </execution>

>

>         </executions>

>

>       </plugin>

>

>       <plugin>

>

>         <groupId>org.apache.maven.plugins</groupId>

>

>         <artifactId>maven-source-plugin</artifactId>

>

>         <executions>

>

>           <execution>

>

>             <id>package-sources</id>

>

>             <phase>package</phase>

>

>             <goals>

>

>               <goal>jar</goal>

>

>               <goal>test-jar</goal>

>

>             </goals>

>

>           </execution>

>

>         </executions>

>

>       </plugin>

>

> First results are very good. For example the MSS-3.3 project that used

> to require 42 minutes for building was built in just 23 minutes.

>

> Thanks,

> Seva

>

> -----Original Message-----

> From: Kohsuke Kawaguchi [mailto:[hidden email]]

> Sent: Monday, May 19, 2008 10:55 AM

> To: [hidden email]

> Subject: Re: Multiple archiving for maven-style projects

>

>

> Any chance you can create a test case for me?

>

> What goal do you execute?

>

> Seva Popov wrote:

>> Hi,

>>

>> Could somebody explain why Hudson executes the archiving so many times

>> for maven-style projects? We are using Hudson version 1.211.

>>

>> For example you can see that that every artifact below is archived 5

>> times during a single build. It seems that this has a huge impact on

> the

>> performance of our maven-style builds. The project converted from

>> maven-style to the free-style is built much, much faster.

>>

>> ..........

>>  [HUDSON] Recording test results

>> [INFO] [jar:jar]

>> [INFO] Building jar:

>> 

> /usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2

>> /a2/target/a2-1.0-SNAPSHOT.jar

>> [INFO] Preparing source:jar

>> [WARNING] Removing: jar from forked lifecycle, to prevent recursive

>> invocation.

>> [HUDSON] Archiving

>> 

> /usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2

>> /a2/pom.xml to

>> 

> /usr/local/hudson/bin/../workspace/jobs/project2/modules/com.tvworks.tva

>> 

> .test$a2/builds/2008-05-14_16-51-52/archive/com.tvworks.tva.test/a2/1.0-

>> SNAPSHOT/pom.xml

>> [HUDSON] Archiving

>> 

> /usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2

>> /a2/target/a2-1.0-SNAPSHOT.jar to

>> 

> /usr/local/hudson/bin/../workspace/jobs/project2/modules/com.tvworks.tva

>> 

> .test$a2/builds/2008-05-14_16-51-52/archive/com.tvworks.tva.test/a2/1.0-

>> SNAPSHOT/a2-1.0-SNAPSHOT.jar

>> [INFO] [enforcer:enforce {execution: enforce-versions}]

>> [INFO] The cached results are still valid. Skipping the rule:

>> org.apache.maven.plugins.enforcer.RequireMavenVersion

>> [INFO] The cached results are still valid. Skipping the rule:

>> org.apache.maven.plugins.enforcer.RequireJavaVersion

>> [HUDSON] Archiving

>> 

> /usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2

>> /a2/pom.xml to

>> 

> /usr/local/hudson/bin/../workspace/jobs/project2/modules/com.tvworks.tva

>> 

> .test$a2/builds/2008-05-14_16-51-52/archive/com.tvworks.tva.test/a2/1.0-

>> SNAPSHOT/pom.xml

>> [HUDSON] Archiving

>> 

> /usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2

>> /a2/target/a2-1.0-SNAPSHOT.jar to

>> 

> /usr/local/hudson/bin/../workspace/jobs/project2/modules/com.tvworks.tva

>> 

> .test$a2/builds/2008-05-14_16-51-52/archive/com.tvworks.tva.test/a2/1.0-

>> SNAPSHOT/a2-1.0-SNAPSHOT.jar

>> [INFO] [source:jar {execution: package-sources}]

>> [INFO] Building jar:

>> 

> /usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2

>> /a2/target/a2-1.0-SNAPSHOT-sources.jar

>> [INFO] Preparing source:test-jar

>> [WARNING] Removing: jar from forked lifecycle, to prevent recursive

>> invocation.

>> [WARNING] Removing: test-jar from forked lifecycle, to prevent

> recursive

>> invocation.

>> [HUDSON] Archiving

>> 

> /usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2

>> /a2/pom.xml to

>> 

> /usr/local/hudson/bin/../workspace/jobs/project2/modules/com.tvworks.tva

>> 

> .test$a2/builds/2008-05-14_16-51-52/archive/com.tvworks.tva.test/a2/1.0-

>> SNAPSHOT/pom.xml

>> [HUDSON] Archiving

>> 

> /usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2

>> /a2/target/a2-1.0-SNAPSHOT.jar to

>> 

> /usr/local/hudson/bin/../workspace/jobs/project2/modules/com.tvworks.tva

>> 

> .test$a2/builds/2008-05-14_16-51-52/archive/com.tvworks.tva.test/a2/1.0-

>> SNAPSHOT/a2-1.0-SNAPSHOT.jar

>> [HUDSON] Archiving

>> 

> /usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2

>> /a2/target/a2-1.0-SNAPSHOT-sources.jar to

>> 

> /usr/local/hudson/bin/../workspace/jobs/project2/modules/com.tvworks.tva

>> 

> .test$a2/builds/2008-05-14_16-51-52/archive/com.tvworks.tva.test/a2/1.0-

>> SNAPSHOT/a2-1.0-SNAPSHOT-sources.jar

>> [INFO] [enforcer:enforce {execution: enforce-versions}]

>> [INFO] The cached results are still valid. Skipping the rule:

>> org.apache.maven.plugins.enforcer.RequireMavenVersion

>> [INFO] The cached results are still valid. Skipping the rule:

>> org.apache.maven.plugins.enforcer.RequireJavaVersion

>> [HUDSON] Archiving

>> 

> /usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2

>> /a2/pom.xml to

>> 

> /usr/local/hudson/bin/../workspace/jobs/project2/modules/com.tvworks.tva

>> 

> .test$a2/builds/2008-05-14_16-51-52/archive/com.tvworks.tva.test/a2/1.0-

>> SNAPSHOT/pom.xml

>> [HUDSON] Archiving

>> 

> /usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2

>> /a2/target/a2-1.0-SNAPSHOT.jar to

>> 

> /usr/local/hudson/bin/../workspace/jobs/project2/modules/com.tvworks.tva

>> 

> .test$a2/builds/2008-05-14_16-51-52/archive/com.tvworks.tva.test/a2/1.0-

>> SNAPSHOT/a2-1.0-SNAPSHOT.jar

>> [HUDSON] Archiving

>> 

> /usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2

>> /a2/target/a2-1.0-SNAPSHOT-sources.jar to

>> 

> /usr/local/hudson/bin/../workspace/jobs/project2/modules/com.tvworks.tva

>> 

> .test$a2/builds/2008-05-14_16-51-52/archive/com.tvworks.tva.test/a2/1.0-

>> SNAPSHOT/a2-1.0-SNAPSHOT-sources.jar

>> [INFO] [source:test-jar {execution: package-sources}]

>> [INFO] Building jar:

>> 

> /usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2

>> /a2/target/a2-1.0-SNAPSHOT-test-sources.jar

>> [INFO] [install:install]

>> [INFO] Installing

>> 

> /usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2

>> /a2/target/a2-1.0-SNAPSHOT.jar to

>> 

> /home/hudson/.m2/repository/com/tvworks/tva/test/a2/1.0-SNAPSHOT/a2-1.0-

>> SNAPSHOT.jar

>> [INFO] Installing

>> 

> /usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2

>> /a2/target/a2-1.0-SNAPSHOT-sources.jar to

>> 

> /home/hudson/.m2/repository/com/tvworks/tva/test/a2/1.0-SNAPSHOT/a2-1.0-

>> SNAPSHOT-sources.jar

>> [INFO] Installing

>> 

> /usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2

>> /a2/target/a2-1.0-SNAPSHOT-test-sources.jar to

>> 

> /home/hudson/.m2/repository/com/tvworks/tva/test/a2/1.0-SNAPSHOT/a2-1.0-

>> SNAPSHOT-test-sources.jar

>> [HUDSON] Archiving

>> 

> /usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2

>> /a2/pom.xml to

>> 

> /usr/local/hudson/bin/../workspace/jobs/project2/modules/com.tvworks.tva

>> 

> .test$a2/builds/2008-05-14_16-51-52/archive/com.tvworks.tva.test/a2/1.0-

>> SNAPSHOT/pom.xml

>> [HUDSON] Archiving

>> 

> /usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2

>> /a2/target/a2-1.0-SNAPSHOT.jar to

>> 

> /usr/local/hudson/bin/../workspace/jobs/project2/modules/com.tvworks.tva

>> 

> .test$a2/builds/2008-05-14_16-51-52/archive/com.tvworks.tva.test/a2/1.0-

>> SNAPSHOT/a2-1.0-SNAPSHOT.jar

>> [HUDSON] Archiving

>> 

> /usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2

>> /a2/target/a2-1.0-SNAPSHOT-sources.jar to

>> 

> /usr/local/hudson/bin/../workspace/jobs/project2/modules/com.tvworks.tva

>> 

> .test$a2/builds/2008-05-14_16-51-52/archive/com.tvworks.tva.test/a2/1.0-

>> SNAPSHOT/a2-1.0-SNAPSHOT-sources.jar

>> [HUDSON] Archiving

>> 

> /usr/local/hudson/workspace/jobs/project2/workspace/project2/subproject2

>> /a2/target/a2-1.0-SNAPSHOT-test-sources.jar to

>> 

> /usr/local/hudson/bin/../workspace/jobs/project2/modules/com.tvworks.tva

>> 

> .test$a2/builds/2008-05-14_16-51-52/archive/com.tvworks.tva.test/a2/1.0-

>> SNAPSHOT/a2-1.0-SNAPSHOT-test-sources.jar

>> [INFO]

>> [INFO]

>> ......

>>

>> Thanks,

>> Seva

>>

>>

>> ---------------------------------------------------------------------

>> To unsubscribe, e-mail: [hidden email]

>> For additional commands, e-mail: [hidden email]

>>

>>

>

>

 

 

--

Kohsuke Kawaguchi

Sun Microsystems                   [hidden email]