Manual Test Authoring through Microsoft Office Infopath 2003

Manual Test Authoring through Microsoft Office Infopath 2003

• GMohr

  Hi Rob,

  Unfortunately, the right contacts to answer your questions on our team are on vacation. As far as I know, our extensibility model does not allow you to extend the manual test itself, so you will have to create your own test type as you suggested. Then instead of calling Shell.Interop.Open, you will have to create your own editor by deriving TuipEditorWindowPane and TuipEditorControl. The trick is however, that the adapter has to wait for completion of the manual test and the existing UI we have there is not accessible by custom TIP implementations. So you will have to implement some UI to be executed from the TIP adapter implementation.

  Regards,

  Boris

  
  
  

• TBJ

  Hi Rob,

  Let me try to address your issues:

  1) & 2)  Yes, your observation is correct. InvokeEditor is only called when you a test is to be opened from one of the Test windows. If you open it from, say, the Solution Explorer, you will get an xml editor, or IE. An alternative way to approach this, is to implement a VS Editor Factory.

  This approach requires that you somehow embed in the xml file (maybe via InfoPath) some unique characteristics, so that given an xml file, you can tell that it is a file that your TIP created, but not some other random xml file. In additional, you need to have a "template" xml file that also contains certain unique data that helps you identify it as a "new" test case of your test type.

  Your editor factory should be registered as a possible editor for "xml" file. When VS calls you to open an xml file, if you detect that this is a file representing your test type's test case, you will then proceed to open it the way you want it to. If it represents a new test case of your test type, you can do something different. If it's none of the above, then you return back to VS telling it to try other editors.

  With this approach, your InvokeEditor would simply call VS to open the file (because VS will eventually invoke your editor factory). Also, the "non-InvokeEditor" paths will also work.

  One additional thing you can consider, is to not save as ".xml" but some unique extension that you choose. This will make it easier for you to identify your test type, and for VS to route the extension to your editor factory.

  How does the user go about creating a new test of your type I've assumed that you have a template that they can pick in the Add new test / Add new item dialog.

  3) My recommendation is to put the execution UI (the code that brings up InfoPath etc). This method can be called from two different code paths: A) When the InvokeResultViewer method is called (user double clicks on your test type's result line in the Results window to view the details of a result), B) When a test of your test type is executed.

  More about (B): Your ultimate goal is that the adapter can tell the UI to be brought up, user fills in the information, and eventually the adapter gets back the result. Here's a possible approach. The adapter and UI live in different processes. You can use .Net remoting IPC channel to get the two to communicate. Start with defining an interface for the adapter to call the UI, say IMyTestTypeExecProxy. The UI side, which has access to the entry point for launching the execution UI, opens an IPC channel and register IMyTestTypeExecProxy. When execution reaches your test, your adapter's Run method gets IMyTestTypeExecProxy by using IPC channel, and calls a method on the interface (say, "RunTest", which returns a test result). IMyTestTypeExecProxy.RunTest's implementation launches the code to bring up the execution UI on the UI thread, and waits on an event for the execution to be finished. The event is to be sent by your execution UI. When the RunTest method finishes waiting, the adapter's Run method is unblocked, and using the test result returned, it can do a testContext.ResultSink.AddResult(result).

  4) Unless you can turn your test type to an automated one (currently it is manual since people need to manually edit the results), I am afraid remote execution can't be supported. If tests need to be run by different people, they will have to be in different runs.

  Extensibility story of a test list includes letting you add new test lists and add tests to a test list etc. The behavior of a test list cannot be modified. If you are interested in creating your own "test suite", the best way to do it would be to implement a test type that contains other tests, similar to our "ordered test" test type.

  Hope this helps. Thanks!

  Winnie

  
  

• Ray_GTI-R

  Hi Rob,

  To support an Infopath file as a test, yes, you will have to provide your own test type implementation. So you are on the right track.

  For the ITuip interface's InvokeEditor method -- are you inheriting from the BaseTuip class   If so, it's InvokeEditor implementation attempts to open the file via DTE (dte.ItemOperations.OpenFile). Does that work / not work for you  

  There are also some additional things you need to consider, which you may have solved already. I thought I'd bring them up anyway:

  • Each test has an associated test id which uniquely identifies the test. This id needs to be persisted with the test. Same applies to the "Owner" and "Description" properties. I am not familiar enough with InfoPath to know if it lets you persist a bag of properties with an InfoPath file. If it does, you can probably achieve this via InfoPath's automation model to load and save these properties.
  • I assume your test type is not an automated one I.e. user needs to explicitly provides a result when "running" the test If so, please make sure you return false for the IsAutomated property of your test element implementation. Also, the following comments would be applicable.
  • You mentioned that you'd like to use InfoPath as the editor of your test type, which makes sense. What about the UI for specifying results   For manual tests, we implement a window that hosts an IE control (in which we load the manual test file), alongside with other controls to specify results (pass/fail, comments).  You might want to do something similar. 
  • As Boris mentioned, there is some synchronization issue you need to handle because of the non-automated nature of your test type. Basically your adapter and your results UI need to communicate, so that the adapter can tell the results UI to start up, and to abort, if such event happens. Your results UI also needs to communicate that the test is "completed" (e.g. a result has been specified).  For our manual test implementation, we use .Net remoting to accomplish this.

  Hope this helps!

  Winnie

   

  
  

• gon

  Ok, I think I have that working now...at least in the Experimental Hive. What do I need to do to deploy it Just run regpkg.exe on "MyEditor.dll" or create a reg file to add the Editor Factory GUID to the HKLM/MS/VS/8.0/Editors Is that all I need to do

  Rob

  
  

• CondonG

  I've been looking at the EditorFactory, but I get the feeling that there must be a better way. I don't want a custom window pane or 99% of the other functionality of my own EditorFactory. That seems like a lot of pointless code just to catch the FileOpen() call.

  I found a way through the InfoPath object model to deal with the distinction between a newly created test case and a previously existing one by just passing the XML and the form.

  Now all I really need is the simplest possible way to catch all possible paths to editing my custom test type. If I have to use my own EditorFactory for this, is there a way to not implement my own window pane, etc. and just override the LoadTest() method I'm assuming there isn't, but it would save me a ton of pretty much wasted effort.

  Rob

   

  
  

• SDHop

  Ok, that makes more sense. I actually did create my own custom editor and sort of got it working, but the approach you are describing sounds much more simple (i.e. - maintainable ) so I'm going to give that a try. I'm sure I'll be back with more questions.

  Rob

  
  

• Polican

  Ok, well let's try something simpler then. Does anyone have any example of using InvokeEditor in a custom test type to launch a different editor

  Since Visual Studio will launch Infopath when you do a file open of a .xsn file, I'd like to just mimic that behavior for editing a test. I'm digging through the documentation on Microsoft.VisualStudio.Shell.Interop for just simply calling an Open() call, but any help, or even just a response would be greatly appreciated.

   

  Rob

   

  
  

• Genyus

  Thanks for the inforomation, Boris.

  I'm actually playing with the EnvDTE object right now to see if I could just use that to call OpenFile() from inside the InvokeEditor method of my custom test type. My next alternative is to just call into the InfoPath object model. I'd really rather not write my own editor. The intent is to use the InfoPath editor either standalone or embedded in the Visual Studio window.

  If you have any thoughts or opinions, I'd love to hear them. Otherwise, I'll keep playing around on this end until people get back after the new year.

  Rob

  
  

• ClausT

  Thanks Winnie,

  That definitely helps. I'll try some of that out and then I'm sure I'll be back with more questions. :)

  Rob

  
  

• leamas

  Thanks Winnie,

  That's all very useful information.

  Basically, what I'd like to do is create a form in Infopath(.xsn file) that will be launched when a test is created or edited. The actual test will be in the form of the .xml file that is output from Infopath when the .xsn form is filled out and saved as .xml. So the TestElement.Storage would be the XML file, not the InfoPath form. The form is more of just a dynamic template to create the XML.

  Then when the test is run, I want to load a different Infopath form for manual execution, that would have fields for Pass/Fail, results of each step, etc. When the test was complete, the results would be saved as .xml output from InfoPath. Then I need to get those results back into Team System so that they are visible via the Test Results window and the Reports.

  I've conquered some of this already, but creating my own custom test type. I used InvokeEditor() in the Tuip to create an automated object (DTE) that calls FileOpen(). I also modified the Load() method in the Tip to read XML from an existing test and set the Name, Storage, and Description properties of my TestElement class. So far so good, but I still have a few issues to work out:

  1) InvokeEditor only seems to be called when you double-click on a test in the Test Manager, but not when a new test is created. So I need to use either the Load() method or some other means of catching the "non-InvokeEditor()" paths to editing a test case.

  2) I need a slightly different path for a new test (load the empty .xsn form) than for editing an existing test case (load the .xml from that test's Storage data and then pass it to the .xsn for so that the test data will be prepopulated).

  3)  If I modify the Run() method of the adapter to launch InfoPath or some IE plugin for test execution, how do I get the results back into Team System in a format that it will understand Do I write out results as XML, or do I have to communicate with a service and pass the results that way I'm a little confused as to exactly how the Run() method communicates it's results back to the ViewResults class. Ideally, I'd like to not have a custom ResultsViewer and just use the Team System viewer and built in Reports.

  4) Then the biggest issue is that I would like to be able to run some of the tests remotely and then somehow just merge the results back into a single run. The idea would be that we kick off a test run of potentially 1000s of tests and several different testers run subsets of the run and somehow communicate those results back into a single run report.

   

  Thanks again, for the help. If you have any suggestions or ideas for any of the last 4 issues, then I'd love to hear it. I am slowly becoming more familiar with the Test Type extensibility. I just wish some of the documentation was a little more detailed. Oh, this might be better addressed in a seperate thread, but I was also wondering about extensibility for the Test Lists. There seems to be lots of documentation on extending test cases, but if I want to modify the behavior of a Test List, or even create my own custom "test suite" type of test list, is that available for extensibility

  Rob

   

   

  
  

• eclipse2k

  Hi Rob,

  I apologize for the late reply.

  I assume by "EditorFactory" you are referring to Microsoft.VisualStudio.TestTools.Vsip.EditorFactory, which is a class provided by our extensibility framework. This class is intended to be used when you do want to implement your own Window Pane to be hosted in VS.

  In your case, it seems that what you want to use is to load InfoPath (as appropriate) outside of VS, not your own Window pane inside VS. Is this correct If so, you just need to offer an editor factory by providing a class that implements the VSIP "IVsEditorFactory" interface (documented in the Visual Studio SDK). The most important method in the class is "CreateEditorInstance". In this method, you can find out whether the passed in file represents a test of your test type, and if so launch the external editor, which is InfoPath. While CreateEditorInstance has "docView" and "docData" etc as "out" parameters, you can just set them to null.

  (i.e. Microsoft.VisualStudio.TestTools.Vsip.EditorFactory is also an implementation of IVsEditorFactory, but since you are not offering an inside-VS window pane, you can by pass EditorFactory class directly, and implement IVsEditorFactory yourself.)

  With this approach, you don't have to implement your own window pane and the "LoadTest" business. As long as your Tuip derives from the BaseTuip class, which already implements InvokeEditor that calls VS to look for your editor factory, there is nothing else you need to do. That's the reason why I recommended creating your own editor factory (NOT by deriving from EditorFactory, but by implementing IVsEditorFactory, as explained above), as that's the catch-all way of ensuring that whenever your test needs to be opened --- whether user has double clicked the file in solution explorer, or has double clicked test in the test view --- the appropriate (external) editor is opened because that's in your editor factory implementation.

  Hope this helps.

  Thanks.

  Winnie

  
  
• Manual Test Authoring through Microsoft Office Infopath 2003