JUnit test reports and notification email

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

JUnit test reports and notification email

Ben Mills
Hello,

Having some trouble with publishing JUnit test reports and the notification email link.
Probably my misunderstanding but I could use some help.

1) My project is running the JUnit tests and building successfully...

test:
    [junit] Running org.cfp.jamestown.test.unit.persistence.AdministratorRepositoryTest
    [junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 4.574 sec
    [junit] Test org.cfp.jamestown.test.unit.persistence.AdministratorRepositoryTest FAILED
    [junit] Running org.cfp.jamestown.test.unit.persistence.ResourceRepositoryTest
    [junit] Tests run: 5, Failures: 0, Errors: 5, Time elapsed: 6.54 sec
    [junit] Test org.cfp.jamestown.test.unit.persistence.ResourceRepositoryTest FAILED
    [junit] Running org.cfp.jamestown.test.unit.persistence.UserRepositoryTest
    [junit] Tests run: 9, Failures: 0, Errors: 9, Time elapsed: 7.567 sec
    [junit] Test org.cfp.jamestown.test.unit.persistence.UserRepositoryTest FAILED
    [junit] Running org.cfp.jamestown.test.unit.service.AdministratorServiceTest
    [junit] Tests run: 6, Failures: 0, Errors: 0, Time elapsed: 0.105 sec
    [junit] Running org.cfp.jamestown.test.unit.service.EmailServiceTest
    [junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 0.186 sec
    [junit] Test org.cfp.jamestown.test.unit.service.EmailServiceTest FAILED
    [junit] Running org.cfp.jamestown.test.unit.service.NewsArticleServiceTest
    [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 0.121 sec
    [junit] Running org.cfp.jamestown.test.unit.service.ResourceServiceTest
    [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 0.113 sec
    [junit] Running org.cfp.jamestown.test.unit.service.UserServiceTest
    [junit] Tests run: 6, Failures: 0, Errors: 0, Time elapsed: 0.105 sec
[junitreport] Processing /opt/apache-tomcat-5.5.23/jobs/JAMES/workspace/jamestown/docs/tests/TESTS-TestSuites.xml to /tmp/null1726576797
[junitreport] Transform time: 5798ms
[junitreport] Deleting: /tmp/null1726576797

BUILD SUCCESSFUL
Total time: 52 seconds
Recording test results
Test reports were found but none of them are new. Did tests run?
Sending e-mails to: [hidden email]
finished: FAILURE

but fails with "Test reports were found but none of them are new. Did tests run?"  
Not sure why it thinks the reports aren't new (I had run ant clean prior to this and the files did not exist).

2)  Also, the notification email includes this link:
 http://<hudson_server>/job/JAMES/ws/jamestown/docs/tests/TESTS-TestSuites.xml

which of course takes me to TESTS-TestSuites.xml, but it displays:
This XML file does not appear to have any style information associated with it. The document tree is shown below.

How is this link created/controlled?  I would like the email to link to formatted test results... The ant target "test" is creating an index.html which renders all the test XML files, and it would be ideal if the email could just link to that... but I don't see how I can control this.  

Thanks.


Reply | Threaded
Open this post in threaded view
|

Re: JUnit test reports and notification email

Kohsuke Kawaguchi
Administrator
Ben Mills wrote:

> Hello,
>
> Having some trouble with publishing JUnit test reports and the  
> notification email link.
> Probably my misunderstanding but I could use some help.
>
> 1) My project is running the JUnit tests and building successfully...
>
> test:
>      [junit] Running  
> org.cfp.jamestown.test.unit.persistence.AdministratorRepositoryTest
>      [junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed:  
> 4.574 sec
>      [junit] Test  
> org.cfp.jamestown.test.unit.persistence.AdministratorRepositoryTest  
> FAILED
>      [junit] Running  
> org.cfp.jamestown.test.unit.persistence.ResourceRepositoryTest
>      [junit] Tests run: 5, Failures: 0, Errors: 5, Time elapsed: 6.54  
> sec
>      [junit] Test  
> org.cfp.jamestown.test.unit.persistence.ResourceRepositoryTest FAILED
>      [junit] Running  
> org.cfp.jamestown.test.unit.persistence.UserRepositoryTest
>      [junit] Tests run: 9, Failures: 0, Errors: 9, Time elapsed:  
> 7.567 sec
>      [junit] Test  
> org.cfp.jamestown.test.unit.persistence.UserRepositoryTest FAILED
>      [junit] Running  
> org.cfp.jamestown.test.unit.service.AdministratorServiceTest
>      [junit] Tests run: 6, Failures: 0, Errors: 0, Time elapsed:  
> 0.105 sec
>      [junit] Running  
> org.cfp.jamestown.test.unit.service.EmailServiceTest
>      [junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed:  
> 0.186 sec
>      [junit] Test  
> org.cfp.jamestown.test.unit.service.EmailServiceTest FAILED
>      [junit] Running  
> org.cfp.jamestown.test.unit.service.NewsArticleServiceTest
>      [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed:  
> 0.121 sec
>      [junit] Running  
> org.cfp.jamestown.test.unit.service.ResourceServiceTest
>      [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed:  
> 0.113 sec
>      [junit] Running org.cfp.jamestown.test.unit.service.UserServiceTest
>      [junit] Tests run: 6, Failures: 0, Errors: 0, Time elapsed:  
> 0.105 sec
> [junitreport] Processing /opt/apache-tomcat-5.5.23/jobs/JAMES/
> workspace/jamestown/docs/tests/TESTS-TestSuites.xml to /tmp/
> null1726576797
> [junitreport] Loading stylesheet jar:file:/usr/local/apache-ant-1.7.0/
> lib/ant-junit.jar!/org/apache/tools/ant/taskdefs/optional/junit/xsl/
> junit-frames.xsl
> [junitreport] Transform time: 5798ms
> [junitreport] Deleting: /tmp/null1726576797
>
> BUILD SUCCESSFUL
> Total time: 52 seconds
> Recording test results
> Test reports were found but none of them are new. Did tests run?
> Sending e-mails to: [hidden email]
> finished: FAILURE
>
> but fails with "Test reports were found but none of them are new. Did  
> tests run?"
> Not sure why it thinks the reports aren't new (I had run ant clean  
> prior to this and the files did not exist).
In my environement, this usually means the job run on a slave and its
clock is out of sync.

I should improve the error message to cite at least one file and show
the timestamp.

> 2)  Also, the notification email includes this link:
>   http://<hudson_server>/job/JAMES/ws/jamestown/docs/tests/TESTS-
> TestSuites.xml
>
> which of course takes me to TESTS-TestSuites.xml, but it displays:
> This XML file does not appear to have any style information  
> associated with it. The document tree is shown below.

That's a mozilla message, and you should then see the raw XML.

> How is this link created/controlled?  I would like the email to link  
> to formatted test results... The ant target "test" is creating an  
> index.html which renders all the test XML files, and it would be  
> ideal if the email could just link to that... but I don't see how I  
> can control this.

The e-mail really just contains the last N lines of the build log, and
convert any file paths in there to hyperlinks.



--
Kohsuke Kawaguchi
Sun Microsystems                   [hidden email]

smime.p7s (4K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: JUnit test reports and notification email

Ben Mills


On Apr 30, 2007, at 9:21 PM, Kohsuke Kawaguchi wrote:

Ben Mills wrote:
Hello,
Having some trouble with publishing JUnit test reports and the  notification email link.
Probably my misunderstanding but I could use some help.
1) My project is running the JUnit tests and building successfully...
test:
     [junit] Running  org.cfp.jamestown.test.unit.persistence.AdministratorRepositoryTest
     [junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed:  4.574 sec
     [junit] Test  org.cfp.jamestown.test.unit.persistence.AdministratorRepositoryTest  FAILED
     [junit] Running  org.cfp.jamestown.test.unit.persistence.ResourceRepositoryTest
     [junit] Tests run: 5, Failures: 0, Errors: 5, Time elapsed: 6.54  sec
     [junit] Test  org.cfp.jamestown.test.unit.persistence.ResourceRepositoryTest FAILED
     [junit] Running  org.cfp.jamestown.test.unit.persistence.UserRepositoryTest
     [junit] Tests run: 9, Failures: 0, Errors: 9, Time elapsed:  7.567 sec
     [junit] Test  org.cfp.jamestown.test.unit.persistence.UserRepositoryTest FAILED
     [junit] Running  org.cfp.jamestown.test.unit.service.AdministratorServiceTest
     [junit] Tests run: 6, Failures: 0, Errors: 0, Time elapsed:  0.105 sec
     [junit] Running  org.cfp.jamestown.test.unit.service.EmailServiceTest
     [junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed:  0.186 sec
     [junit] Test  org.cfp.jamestown.test.unit.service.EmailServiceTest FAILED
     [junit] Running  org.cfp.jamestown.test.unit.service.NewsArticleServiceTest
     [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed:  0.121 sec
     [junit] Running  org.cfp.jamestown.test.unit.service.ResourceServiceTest
     [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed:  0.113 sec
     [junit] Running org.cfp.jamestown.test.unit.service.UserServiceTest
     [junit] Tests run: 6, Failures: 0, Errors: 0, Time elapsed:  0.105 sec
[junitreport] Processing /opt/apache-tomcat-5.5.23/jobs/JAMES/ workspace/jamestown/docs/tests/TESTS-TestSuites.xml to /tmp/ null1726576797
[junitreport] Loading stylesheet jar:file:/usr/local/apache-ant-1.7.0/ lib/ant-junit.jar!/org/apache/tools/ant/taskdefs/optional/junit/xsl/ junit-frames.xsl
[junitreport] Transform time: 5798ms
[junitreport] Deleting: /tmp/null1726576797
BUILD SUCCESSFUL
Total time: 52 seconds
Recording test results
Test reports were found but none of them are new. Did tests run?
Sending e-mails to: [hidden email]
finished: FAILURE
but fails with "Test reports were found but none of them are new. Did  tests run?"
Not sure why it thinks the reports aren't new (I had run ant clean  prior to this and the files did not exist).

In my environement, this usually means the job run on a slave and its clock is out of sync.

Not using master/slave setup and the system clock is not out of sync.
Any other possibility?

I should improve the error message to cite at least one file and show the timestamp.

Sounds good - thanks.

2)  Also, the notification email includes this link:
  http://<hudson_server>/job/JAMES/ws/jamestown/docs/tests/TESTS- TestSuites.xml
which of course takes me to TESTS-TestSuites.xml, but it displays:
This XML file does not appear to have any style information  associated with it. The document tree is shown below.

That's a mozilla message, and you should then see the raw XML.

Yes, that's understood, but I guess I'm not seeing the value of this feature.  Our ant script is outputting the JUnit resuts to various XML files and then junit-frames.xsl is used to display them.  Apologies for any confusion on my part, but I was just wanting the notification email to contain a link to the HTML file so developers can easily see the JUnit results with formatting, etc.  Is that possible?

How is this link created/controlled?  I would like the email to link  to formatted test results... The ant target "test" is creating an  index.html which renders all the test XML files, and it would be  ideal if the email could just link to that... but I don't see how I  can control this.

The e-mail really just contains the last N lines of the build log, and convert any file paths in there to hyperlinks.

OK, understood.



-- 
Kohsuke Kawaguchi
Sun Microsystems                   [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: JUnit test reports and notification email

Ben Mills


On May 1, 2007, at 2:44 PM, Ben Mills wrote:



On Apr 30, 2007, at 9:21 PM, Kohsuke Kawaguchi wrote:

Ben Mills wrote:
Hello,
Having some trouble with publishing JUnit test reports and the  notification email link.
Probably my misunderstanding but I could use some help.
1) My project is running the JUnit tests and building successfully...
test:
     [junit] Running  org.cfp.jamestown.test.unit.persistence.AdministratorRepositoryTest
     [junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed:  4.574 sec
     [junit] Test  org.cfp.jamestown.test.unit.persistence.AdministratorRepositoryTest  FAILED
     [junit] Running  org.cfp.jamestown.test.unit.persistence.ResourceRepositoryTest
     [junit] Tests run: 5, Failures: 0, Errors: 5, Time elapsed: 6.54  sec
     [junit] Test  org.cfp.jamestown.test.unit.persistence.ResourceRepositoryTest FAILED
     [junit] Running  org.cfp.jamestown.test.unit.persistence.UserRepositoryTest
     [junit] Tests run: 9, Failures: 0, Errors: 9, Time elapsed:  7.567 sec
     [junit] Test  org.cfp.jamestown.test.unit.persistence.UserRepositoryTest FAILED
     [junit] Running  org.cfp.jamestown.test.unit.service.AdministratorServiceTest
     [junit] Tests run: 6, Failures: 0, Errors: 0, Time elapsed:  0.105 sec
     [junit] Running  org.cfp.jamestown.test.unit.service.EmailServiceTest
     [junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed:  0.186 sec
     [junit] Test  org.cfp.jamestown.test.unit.service.EmailServiceTest FAILED
     [junit] Running  org.cfp.jamestown.test.unit.service.NewsArticleServiceTest
     [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed:  0.121 sec
     [junit] Running  org.cfp.jamestown.test.unit.service.ResourceServiceTest
     [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed:  0.113 sec
     [junit] Running org.cfp.jamestown.test.unit.service.UserServiceTest
     [junit] Tests run: 6, Failures: 0, Errors: 0, Time elapsed:  0.105 sec
[junitreport] Processing /opt/apache-tomcat-5.5.23/jobs/JAMES/ workspace/jamestown/docs/tests/TESTS-TestSuites.xml to /tmp/ null1726576797
[junitreport] Loading stylesheet jar:file:/usr/local/apache-ant-1.7.0/ lib/ant-junit.jar!/org/apache/tools/ant/taskdefs/optional/junit/xsl/ junit-frames.xsl
[junitreport] Transform time: 5798ms
[junitreport] Deleting: /tmp/null1726576797
BUILD SUCCESSFUL
Total time: 52 seconds
Recording test results
Test reports were found but none of them are new. Did tests run?
Sending e-mails to: [hidden email]
finished: FAILURE
but fails with "Test reports were found but none of them are new. Did  tests run?"
Not sure why it thinks the reports aren't new (I had run ant clean  prior to this and the files did not exist).

In my environement, this usually means the job run on a slave and its clock is out of sync.

Not using master/slave setup and the system clock is not out of sync.
Any other possibility?

Upgraded to Hudson 1.105 and used *.xml (rather than specifying the main test suite xml file) in 
the Test report xmls field and all is working fine now... formatted reports are available and
trend graph is working, etc...


I should improve the error message to cite at least one file and show the timestamp.

Sounds good - thanks.

2)  Also, the notification email includes this link:
  http://<hudson_server>/job/JAMES/ws/jamestown/docs/tests/TESTS- TestSuites.xml
which of course takes me to TESTS-TestSuites.xml, but it displays:
This XML file does not appear to have any style information  associated with it. The document tree is shown below.

That's a mozilla message, and you should then see the raw XML.

Yes, that's understood, but I guess I'm not seeing the value of this feature.  Our ant script is outputting the JUnit resuts to various XML files and then junit-frames.xsl is used to display them.  Apologies for any confusion on my part, but I was just wanting the notification email to contain a link to the HTML file so developers can easily see the JUnit results with formatting, etc.  Is that possible?

Obviously, yes it is :)  Please disregard this - I see the value now (per above).


How is this link created/controlled?  I would like the email to link  to formatted test results... The ant target "test" is creating an  index.html which renders all the test XML files, and it would be  ideal if the email could just link to that... but I don't see how I  can control this.

The e-mail really just contains the last N lines of the build log, and convert any file paths in there to hyperlinks.

OK, understood.



-- 
Kohsuke Kawaguchi
Sun Microsystems                   [hidden email]


Reply | Threaded
Open this post in threaded view
|

Re: JUnit test reports and notification email

Kohsuke Kawaguchi
Administrator
In reply to this post by Ben Mills
2007/5/1, Ben Mills <[hidden email]>:
> Yes, that's understood, but I guess I'm not seeing the value of this
> feature.  Our ant script is outputting the JUnit resuts to various XML files
> and then junit-frames.xsl is used to display them.  Apologies for any
> confusion on my part, but I was just wanting the notification email to
> contain a link to the HTML file so developers can easily see the JUnit
> results with formatting, etc.  Is that possible?

It's desirable for the notifications in general to have more useful
content than just the console output dump. So I agree it's an area
that we should look into.

Exactly how we do it, I'm less sure.

--
Kohsuke Kawaguchi

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]