MSTest unexpected behavior - tests pass when they should fail

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

MSTest unexpected behavior - tests pass when they should fail

TomsBrother
Hi All,I have a working environment which is Hudson integrated with SVN and Visual Studio 2010 Pro. I have written 2 unit tests. One of them will always fail. I am getting unexpected test results from Hudson, namely 2 passes. I should be getting 1 pass and 1 fail. The repro : Hudson Build [2 passes] Console Output: c:\hudson\home\jobs\SQLTemplateDemo\workspace>"C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\MSTest.exe" /testcontainer:.\TestProject1\bin\Release\AddProcTests.dll /resultsfile:TestSuiteResults.trx Microsoft (R) Test Execution Command Line Tool Version 10.0.30319.1 Copyright (c) Microsoft Corporation. All rights reserved. Loading .\TestProject1\bin\Release\AddProcTests.dll... Starting execution... Results Top Level Tests ------- --------------- Passed TestProject1.UnitTest1._AddProcedure2Numbers_Test Passed TestProject1.UnitTest1.Complex_Test 2/2 test(s) Passed Summary ------- Test Run Completed. Passed 2 --------- Total 2 Results file: c:\hudson\home\jobs\SQLTemplateDemo\workspace\TestSuiteResults.trx Test Settings: Default Test Settings c:\hudson\home\jobs\SQLTemplateDemo\workspace>exit 0 Processing tests results in file TestSuiteResults.trx Finished: SUCCESS Manually Running the test : [1 pass and 1 fail as expected] C:\hudson\home\jobs\SQLTemplateDemo\workspace>del TestSuiteResults.trx C:\hudson\home\jobs\SQLTemplateDemo\workspace>"C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\MSTest.ex e" /testcontainer:.\TestProject1\bin\Release\AddProcTests.dll /resultsfile:TestSuiteResults.trx Microsoft (R) Test Execution Command Line Tool Version 10.0.30319.1 Copyright (c) Microsoft Corporation. All rights reserved. Loading .\TestProject1\bin\Release\AddProcTests.dll... Starting execution... Results Top Level Tests ------- --------------- Failed TestProject1.UnitTest1._AddProcedure2Numbers_Test Passed TestProject1.UnitTest1.Complex_Test 1/2 test(s) Passed, 1 Failed Summary ------- Test Run Failed. Failed 1 Passed 1 --------- Total 2 Results file: C:\hudson\home\jobs\SQLTemplateDemo\workspace\TestSuiteResults.trx Test Settings: Default Test Settings C:\hudson\home\jobs\SQLTemplateDemo\workspace> Can anyone help me out here ? Any ideas greatly appreciated ...
Reply | Threaded
Open this post in threaded view
|

MSTest unexpected behavior - tests pass when they should fail

TomsBrother

I have a working environment which is Hudson integrated with SVN and Visual Studio 2010 Pro.

I have written 2 unit tests. One of them will always fail.

I am getting unexpected test results from Hudson, namely 2 passes. I should be getting 1 pass and 1 fail.

The repro :

 

Hudson Build  [2 passes] Console Output :

c:\hudson\home\jobs\SQLTemplateDemo\workspace>"C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\MSTest.exe" /testcontainer:.\TestProject1\bin\Release\AddProcTests.dll /resultsfile:TestSuiteResults.trx

Microsoft (R) Test Execution Command Line Tool Version 10.0.30319.1

Copyright (c) Microsoft Corporation. All rights reserved.

   

Loading .\TestProject1\bin\Release\AddProcTests.dll...

Starting execution...

 

Results               Top Level Tests

-------               ---------------

Passed                TestProject1.UnitTest1._AddProcedure2Numbers_Test

Passed                TestProject1.UnitTest1.Complex_Test

2/2 test(s) Passed

 

Summary

-------

Test Run Completed.

  Passed  2

  ---------

  Total   2

Results file:  c:\hudson\home\jobs\SQLTemplateDemo\workspace\TestSuiteResults.trx

Test Settings: Default Test Settings

 

c:\hudson\home\jobs\SQLTemplateDemo\workspace>exit 0

Processing tests results in file TestSuiteResults.trx

Finished: SUCCESS

 

Manually Running the test :  [1 pass and 1 fail as expected]

C:\hudson\home\jobs\SQLTemplateDemo\workspace>del TestSuiteResults.trx

C:\hudson\home\jobs\SQLTemplateDemo\workspace>"C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\MSTest.ex

e" /testcontainer:.\TestProject1\bin\Release\AddProcTests.dll /resultsfile:TestSuiteResults.trx

Microsoft (R) Test Execution Command Line Tool Version 10.0.30319.1

Copyright (c) Microsoft Corporation. All rights reserved.

 

Loading .\TestProject1\bin\Release\AddProcTests.dll...

Starting execution...

 

Results               Top Level Tests

-------               ---------------

Failed                TestProject1.UnitTest1._AddProcedure2Numbers_Test

Passed                TestProject1.UnitTest1.Complex_Test

1/2 test(s) Passed, 1 Failed

 

Summary

-------

Test Run Failed.

  Failed  1

  Passed  1

  ---------

  Total   2

Results file:  C:\hudson\home\jobs\SQLTemplateDemo\workspace\TestSuiteResults.trx

Test Settings: Default Test Settings

 

C:\hudson\home\jobs\SQLTemplateDemo\workspace>

 

 

 

Can anyone help me out here ? Any ideas greatly appreciated ...