Any way to distinguish the test results of same test cases on same build against different platforms

Any way to distinguish the test results of same test cases on same build against different platforms

• Joao.Pio

  Hi Leon,

  Let me try to answer your question here. You can build your code under test into different platform/flavor combinations by specifying the platform and flavor when you create your build type. You can actually specify more than one of such combinations, like Release/Any CPU, or Debug/x86. You can specify running tests against these combinations. Then when you create the actual build, the tests would be running against each of these builds and results will be published.

  Is this what you're asking for

  Thanks,

  James

  VSTS

  
  

• KevinC123

  David,

  Thank you for your reply. You make it clear and save much of my time, because I may investigate it further if I do not understand current design in VSTS.

  Here are my two cents about the possible feature testers ususally want. I suggest to coin a concept which means a set of test records/execution records/test log under Build. The test records are different concept with test cases, though they may share the test steps.

  -- Test records are created by demand by selecting the test cases before execution

  -- Test records are associated with a build. A build could have several set of test records.

  -- Test executor just runs testing according to the test records.

  ....

  We could enrich the test feature a lot if we have this concept. We may call this concept "Test Cycle", or any one you like.

  Leon

  
  
  

• Kodo

  Hi Leon,

  We've run into the same issue in our testing, yet it is feedback we rarely hear from customers. I'm really glad to hear someone else report the same problem and I will be happy to use it to promote the feature.

  James' reply is based on what we've done internally to work with the features of the system to get what we want. Instead of building the product multiple times, what we've done is set up dummy builds with descriptive names of the testing matrix we need. That allows us to publish the results of a run on a different operating system or flavor of the product to a different location which can be tracked seperately from the other runs.

  To do this, we set up another build but do not build the full product. We select a trivial project in the system to build since you must select something.

  Now the build we publish to has the same date suffix (i.e. 20050411.1) but a different prefix, the descriptive one I gave it to identify the test matrix.

  James also suggests using one additional build but with more flavors to distinguish a large test matrix, instead of having many different build names. That cuts down on the clutter in the Team Build tree.

  I'm sorry we don't have the feature working the way you'd expect, but I hope this work around is acceptable for now.

  Cheers,

  David Williamson

  
  
  

• jossy

  James,

  Thank you for your reply.

  But this is not my scenario. Currently, we physically have only ONE actual build (executable code), and we will run the test cases with different environment/configurations. The different environment/configuration(Windows XP or Windows 2003, different motherboards, ...etc) will not affect the actual build.

  We do not think it is a good idea to create the same build several times for different environment/configurations. Is there any alternative solution that could help us out

  Thanks,

  Leon

  
  
  
• Any way to distinguish the test results of same test cases on same build against different platforms