Sorry if this is a repeat - I couldn't find anything though.
We have what I thought would be a pretty common scenario - we have a couple of projects containing our automated unit tests and we want to run these as part of the build. In normal unit testing behavior, we don't want to maintain any lists of the tests, we just want to say "run the tests in this assembly". On the desktop, that's very straight foward, we just throw out a call to "mstest.exe /testcontainer:MyAssembly.dll" However, there's a big brick wall between that point and the server.
- First, it seems that MSTest.exe is not installed as part of either team foundation server or the Team Build installations. (Meaning my build targets can't execute it as a task)
- Second, the TestToolsTask that Team Build uses to execute tests during a build only exposes the ability to build through the test lists (.vsmdi). (Meaning my developers have to all have the tester edition - not feasible)
- Third, the shared code that is used by both mstest.exe and Microsoft.VisualStudio.QualityTools.MSBuildTasks.dll is all bundled up in a tidy little network of assemblies, each friendly with the others, but with every useful class and method for extension marked internal (See InternalsVisibleTo assembly attribute). (Meaning I can't write my own Custom Task to handle this).
All of this combines to frustrate me to no end, as it seems the most basic, required practices of our development process (build & test w/ Continuous integration) is not feasible.
So, my question is: What have I missed I can't imagine that Microsoft missed this standard use case.
Running Assembly-based tests with build server.
Juraj Borza
Thanks for the response Steve...
I'm coming to the same realization... I'm going to have to break down and put the developmen environment on the build server... or more likely define the build server as the various high-end developer machines that are floating around with VS already installed - they don't need all their cycles ;)
I'm completely giving up on the Test List concept until somebody (maybe me ) writes a tool that is able to generate them using the "All Tests in the assembly" concept. I can't even figure out how their schema works, to be honest... we've got multiple test methods with the same names like "ShouldContainCorrectColumns" in our database stored proc tests across different classes and they don't appear to have any differences in the test list.
As for the API... it's pretty well-written and completely locked down with repect to outside use - VERY frustrating.
(btw - looking forward to working with you on the Version Tree plug-in on codeplex)
Eric
Wolf-Michael Haberichter
Eric,
I can help wit a bit of this.
First off, you need to have either VS for Developers, Testers, or Suite on the build server to run the tests.
Second, I utilize a single copy of VS for Testers to create the .vsmdi files for each solution that I build. It is a pain, but it works.
Third, sorry, but I haven't had a chance to get too deep into the API.
I'm also doing CI with MSBuild and would like to hear how things are going for you and how you solve this problem.
-Steve