VS 2005 Integration with MSBuild

Let me ask the question and then provide some background...

How much can we customize an initially VS 2005 generated .csproj msbuild project file without "breaking" the VS 2005 <-> MSBuild integration

Background...

Forgive me if this this overly general question has been answered elsewhere, but I didn't see anything. I have just recently begun investigations related to migrating our existing NAnt build scripts to MSBuild for when we move to .NET 2.0. In our current system, we ask developers to maintain the .csproj files and the nant build files. However, only the nant build files are really important. Our build server runs off of the nant build files. The .csproj files are only for developer usage and we even have VS 2003 tricked into calling out to nant to do compiles (so the .csproj files are primarily just to have the correct list of files in the solution explorer for easy navigation).

Our nant build files do a lot more than just compile the source code. We have things broken up into a seperate build file per c# assembly (aka project). In addition to compiling all the sub-build files in the proper order, the master nant script has targets to do things like delay signing assemblies, building our installer, deploying files, etc.

I'm getting to the question, I promise :)

I can imagine working out most of these ancillary tasks by creating a master msbuild project file that does all the extra stuff and calls <MSBuild> on the .sln file to do the general compiling. However, this can probably only be used to house things that are "build lab" specific (such as deploying final files to internal qa servers) because VS 2005 will not know about this master project file when a developer chooses 'Build' on a solution.

Our goal would be (in contrast to what we have today with nant) to allow the developer to do "normal" things in VS and have them work "normally". So, when they add a file to a C# project, it needs to be added to the msbuild project file. When they choose 'Build' it needs to do the correct build steps.

To accomplish all of our "extra" tasks, I'm imagining having to customize the .csproj files with things like BeforeBuild and AfterBuild targets, extra Property definitions, maybe even making some Property values $Configuration dependant even though the didn't start out that way, etc. So the question comes back to how much can I manipulate the .csproj file (in legal MSBuild ways) without "confusing" VS 2005. I don't want to break VS 2005's ability to add/remove files, references etc, and ideally, I don't want to break VS 2005's ability to manipulate the properies that it knows how to manipulate.

Answer this question

VS 2005 Integration with MSBuild

Milen Lazarov MSFT

DanMoseley wrote:

Configurations and Platforms< xml:namespace prefix = o ns = "urn:schemas-microsoft-com:office:office" />
Configurations are represented in MSBuild projects by properties grouped in a <PropertyGroup> element that has a condition on it. < xml:namespace prefix = st1 ns = "urn:schemas-microsoft-com:office:smarttags" />Visual Studio looks at these conditions in order to create a list of project configurations and platforms to display. To successfully extract this list the conditions must have a format like these examples:

Condition=" '$(Configuration)|$(Platform)' == 'Debug|AnyCPU' "

Condition=" '$(Configuration)' == 'Release' "

Condition=" '$(Something)|$(Configuration)|$(SomethingElse)' == 'xxx|Debug|yyy' "

Visual Studio looks at the conditions on <PropertyGroup>, <ItemGroup>, and <Import> elements for this purpose, and on individual property and item elements.

What exactly doesn't work if this is not respected I've noticed the VS picks anything that has Condition="'$(Platform)' == 'xxxx'" and put's it on the platform list. The same for Configuration.

So, instead of this:

<PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Debug|AnyCPU' ">
<Property1A/>
  <Property2A/>
  <Property3A/>
  <Property4a/>
  <Property5a/>
  <Property6a/>
</PropertyGroup>
<PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Release|AnyCPU' ">
<Property1A/>
  <Property2A/>
  <Property3A/>
  <Property4b/>
  <Property5b/>
  <Property6b/>
</PropertyGroup>
<PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Debug|OtherPlatform' ">
<Property1B/>
  <Property2B/>
  <Property3B/>
  <Property4a/>
  <Property5a/>
  <Property6a/>
</PropertyGroup>
<PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Release|OtherPlatform' ">
<Property1B/>
  <Property2B/>
  <Property3B/>
  <Property4b/>
  <Property5b/>
  <Property6b/>
</PropertyGroup>

can I instead use this:

<PropertyGroup Condition=" '$(Platform)' == 'AnyCPU' ">
<Property1A/>
  <Property2A/>
  <Property3A/>
</PropertyGroup>
<PropertyGroup Condition=" '$(Platform)' == 'OtherPlatform' ">
<Property1B/>
  <Property2B/>
  <Property3B/>
</PropertyGroup>
<PropertyGroup Condition=" '$(Configuration)' == 'Debug' ">
  <Property4a/>
  <Property5a/>
  <Property6a/>
</PropertyGroup>
<PropertyGroup Condition=" '$(Configuration)' == 'Release' ">
  <Property4b/>
  <Property5b/>
  <Property6b/>
</PropertyGroup>

Best regards,
Gustavo Guerra

SanK
Scott writes:

    Is there any way to add support for xxproj so that I can edit
    it in VS 2005 the same way I could a csproj file For example
    a .booproj file for building boo based programs.

VS has many different project systems installed, and out of the box, only three of the installed project systems support MSBuild format project files. They are the C# project system, the VB project system, and the VJ# project system. Through VS extensibility, you can certainly write your own project system for a different programming language and register it, but that's a non-trivial amount of work. So aside from that, you're basically stuck with loading one of the 3 existing project systems. Each of these project systems has an association with a particular project file extension.  The mapping is:

   Extension   Project system          Project system GUID
   .CSPROJ      C# project system        FAE04EC0-301F-11D3-BF4B-00C04F79EFBC
   .VBPROJ      VB project system        F184B08F-C81C-45F6-A57F-5ABD9991F28F
   .VJSPROJ     VJ# project system      E6FDF86B-F3D1-11D4-8576-0002A516ECE8

Really, the only times I can think of when the above mapping becomes relevant is when 1.) you are opening a project file from disk that doesn't have an associated .SLN file, or 2.) when you are adding an existing project to already open Solution. During these two operations, VS looks at the file extension for the project file you are trying to open, looks in the mapping table above, and thereby figures out which project system should be responsible for loading that project. Then it instantiates that project system.

So, if you wanted to name your project file with the .BOOPROJ extension, you've got to decide which project system you want to associate it with -- C#, VB, or VJ# If you wanted to associate .BOOPROJ with the C# project system, what you need to do is add the following registry key:

Windows Registry Editor Version 5.00

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\VisualStudio\8.0\Projects\{FAE04EC0-301F-11d3-BF4B-00C04F79EFBC}]
"PossibleProjectExtensions"="csproj;booproj"

This way, when you ask VS to open a .BOOPROJ project file, it will load up the C# project system (since "FAE04EC0-..." is the GUID for the C# project system). Note you will get a C#-centric experience (intellisense, etc.) for your project file, regardless of what kinds of files you actually have in your .BOOPROJ project file.

Hope that helps ...

--Rajeev
Bilal Haidar - MVP
Well, it probably is fine then My list wasn't necessarily comprehensive..

Dan
R_Bohning
While it isn't specific to VS integration, something else that is missing from the documentation that someone asked me about recently is this comment in the bottom of every VB and C# project:

<Import Project="$(MSBuildBinPath)\Microsoft.CSharp.targets" />


What is the full list of such "magic" names Here's the full list, pulled from microsoft.common.targets:

AfterBuild
AfterClean
AfterCompile
AfterPublish
AfterRebuild
AfterResGen
AfterResolveReferences
BeforeBuild
BeforeClean
BeforeCompile
BeforePublish
BeforeRebuild
BeforeResGen
BeforeResolveReferences

To use any of these, just define a target with the appropriate magic name and put it below the <Import> tag in your project. Inside the target, use any MSBuild tasks (ours or custom ones) and they will run at the appropriate time.

All of these were created to make it unnecessary in the basic case to add something like this below the import:
<PropertyGroup>
<ResGenDependsOn>$(ResgenDependsOn);MyTargetName</ResGenDependsOn>
</PropertyGroup>
<Target Name="MyTargetName">
...
</Target>

We just did the work for you for some common cases. However, you'll still want to use the "dependsOn" way if you want to have maximum control over where the target goes. Another reason for "dependsOn" is if you want to add more than one target: for example, you have common targets that add a target before ResGen, and you want projects to be able to add a target there as well.
RamanaR
Kevin,
This feature didn't make beta 2, but it might be in the July CTP depending on when that was cut.. I'm not able to check right now. It will be in the final build though.

In beta 2, after unloading the project, you have to go to File>Open File, and go choose the project file, then it will open in the editor. (Even that's still better than in VS 2003, where it would try and out-think you and load the project up instead... you had to explicitly do Open With and choose the XML editor to get around that)

In the next version, we'd like to make this less clunky still. It shouldn't need 4 clicks. You shouldn't have to unload the project to edit it, anyway. But for VS 2005, it was getting late so we could only make this small improvement.

Dan
JeanH
Out of curiosity - why would you want to do this

Faisal
Anna Elise
Great info, Dan. Thanks!

One question:
>"Unload Project". Then right click again on the project and choose
> "Edit Project".

I don't see an Edit Project item on the context menu after unloading the project. Is that a limitation in Beta 2

Dawn2005

 DanMoseley wrote:

Project file extension
MSBuild.exe recognizes any project file extension matching the pattern .*proj. However < xml:namespace prefix = st1 ns = "urn:schemas-microsoft-com:office:smarttags" />Visual Studio only recognizes a subset of these project file extensions because it uses the extension to choose which language-specific project system to load the project with. Visual Studio does not have a language-neutral MSBuild based project system. 
< xml:namespace prefix = o ns = "urn:schemas-microsoft-com:office:office" /> 
For example, projects that have the extension .csproj will be loaded by the C# project system, but a project with the extension .xxproj cannot be loaded by Visual Studio. A project file for source files in an arbitrary language must use the same extension as VB, C#, or J# project files to be loaded in Visual Studio.

Is there any way to add support for xxproj so that I can edit it in VS 2005 the same way I could a csproj file For example a .booproj file for building boo based programs.

Scott

R2B2
Well for some time, VS would confuse the Defines of each Platform/Configuration, but then I must have changed something else and it now it gets it right again, so...
Christian N

Just a clarification of what Faisal said -- you can certainly do this kind of thing to insert your target into the build:

<PropertyGroup>
   <CoreCompileDependsOn>
      $(CoreCompileDependsOn);
      MyCustomTarget
   </CoreCompileDependsOn>
</PropertyGroup>

but it's not the easiest way. The easiest way is to override the dummy, empty targets we've sprinkled throughout the build process that are already inserted into the chain. These have names like BeforeBuild, AfterBuild, BeforeCompile, AfterCompile and so on.
=============

As for your original question: how much can you modify your project and still have it work with VS The answer should be, a lot. VS is just a host of MSBuild like msbuild.exe is. Here's some of the assumptions VS does depend on though. I am going to try to make up a full list and post back shortly...

Dan
mr vice
Awesome question

Being able to fully customize your build process is one of the overarching goals of MSBuild. The build process used by VS is entirely available for you to see, use and tweak to your liking. This means that you can tweak your VS build process to your liking. In fact, if you look at the targets file - Microsoft.Common.targets (which is imported by either Microsoft.VisualBasic.targets or Microsoft.CSharp.targets), you will see that there you have a lot more control than just pre-build and post-build steps.

Microsoft.Common.targets contains almost all of the build process used by Visual Studio for building managed projects. Here's an example of how I have customized my build by adding a custom target into the build process just before my "CoreCompile" target runs.

<PropertyGroup>
   <CoreCompileDependsOn>
      $(CoreCompileDependsOn);
      MyCustomTarget
   </CoreCompileDependsOn>
</PropertyGroup>

This is just an example of the most straightforward way to plug in your own target into the build process. Simply find the property that gets used for the DependsOn property for the target before which you'd like to run your custom step, and add your own target to the list of dependent targets. And in your target, you can do interesting things like do certain things only when specific property values are passed in, as might be the case if you are running build-lab specific stuff that doesn't run in the IDE.

There is only one thing to keep in mind - i.e. for customized projects you will see a security dialog show up asking you whether the project you are opening is a trusted one. This is to make sure that there is no automatic code execution upon launching an unknown project. Once you go through the process and tell VS to not ask you everytime you open that project, you should be ok.

I hope I have answered your question - take a look through the targets file and let us know if all of this makes sense.

Thanks.
Faisal Mohamood
MSBuild Team
Baerin
OK, here's my writeup of VS integration notes. I'm going to try to get this into the official docs, too. Anything missing from here that you need, let me know!

Dan

==========

< xml:namespace prefix = st1 ns = "urn:schemas-microsoft-com:office:smarttags" />Visual Studio integration with MSBuild< xml:namespace prefix = o ns = "urn:schemas-microsoft-com:office:office" />

Overview

Visual Studio 2005 hosts MSBuild to load and build any managed projects. Because MSBuild is responsible for the project, almost any MSBuild format project can be successfully used in Visual Studio, even the project it was authored by a different tool and has a completely customized build process.

This topic describes specific aspects of Visual Studio's MSBuild hosting that should be considered when customizing projects and targets files that you wish to load and build in Visual Studio. These will help you make sure Visual Studio features like intellisense and debugging work for your custom project.

Project file extension

MSBuild.exe recognizes any project file extension matching the pattern .*proj. However Visual Studio only recognizes a subset of these project file extensions because it uses the extension to choose which language-specific project system to load the project with. Visual Studio does not have a language-neutral MSBuild based project system.

For example, projects that have the extension .csproj will be loaded by the C# project system, but a project with the extension .xxproj cannot be loaded by Visual Studio. A project file for source files in an arbitrary language must use the same extension as VB, C#, or J# project files to be loaded in Visual Studio.

Well-known target names

Choosing the "Build" command in Visual Studio will execute the default target in the project. Often, this target is also named "Build". Choosing the "Rebuild" or "Clean" command will attempt to execute a target of the same name in the project. Choosing "Publish" will execute a target named "PublishOnly" in the project.

Configurations and Platforms

Configurations are represented in MSBuild projects by properties grouped in a <PropertyGroup> element that has a condition on it. Visual Studio looks at these conditions in order to create a list of project configurations and platforms to display. To successfully extract this list the conditions must have a format like these examples:

Condition=" '$(Configuration)|$(Platform)' == 'Debug|AnyCPU' "

Condition=" '$(Configuration)' == 'Release' "

Condition=" '$(Something)|$(Configuration)|$(SomethingElse)' == 'xxx|Debug|yyy' "

Visual Studio looks at the conditions on <PropertyGroup>, <ItemGroup>, and <Import> elements for this purpose, and on individual property and item elements.

Additional Build Actions

Visual Studio allows you to change the item type of a file in a project in the Build Action field of the properties tool window using a drop-down menu. "Compile", "EmbeddedResource", "Content", and "None" item types are always listed in this menu, along with any other item types that are on items already in your project. To ensure any custom item types are always available in this menu, you can add their names to an item list called "AvailableItemName". For example, adding the following to your targets file will add the custom type "JScript" to this menu for all projects that import it:

            <ItemGroup>

                        <AvailableItemName Include="JScript"/>

            </ItemGroup>

Some item types are special to Visual Studio but not normally listed in this dropdown.

In-process compilers

Where possible Visual Studio will attempt to use the in-process versions of the Visual Basic or C# compilers for higher performance. For this to work correctly, the following conditions must be met:

-- there must be a task in any target called "Csc" (for C# projects) or "Vbc" (for Visual Basic projects)

-- the UseHostCompilerIfAvailable parameter must be set to true on the task

-- all parameters on the task are supported by the in-process compiler. This is the list of parameter values on the Csc task that are not supported by the C# in-process compiler:

            -- NoConfig=false or <blank>

            -- ResponseFiles=<non blank>

            -- AdditionalLibPaths=<non blank>

            -- AddModules=<non blank>

            -- CodePage=<non zero>

            -- GenerateFullPaths=true

            -- LinkResources=<non blank>

If these conditions are not met, the compilation will work correctly, but will fall back to using the command-line compiler.

Design-time intellisense

To get intellisense before a build has generated any output assembly, the following conditions must be met:

            -- there must be a target named "Compile"

            -- either this target or one of its dependencies must call the compiler task

            -- either this target or one of its dependencies must cause the compiler to get passed all necessary parameters for intellisense, particularly all references

            -- the conditions listed above for using the in-process compiler must be met

Building Solutions

Within Visual Studio, the solution file and project build ordering is controlled by Visual Studio itself. When building a solution with msbuild.exe on the command line, MSBuild parses the solution file and orders the project builds. In both cases the projects are built individually in dependency order, and project to project references are not traversed. In contrast when individual projects are built with msbuild.exe, project to project references are traversed.

When building inside Visual Studio, the property $(BuildingInsideVisualStudio) is set to true. This can be used in your project or targets files to cause the build to behave differently.

Display of Properties and Items

Visual Studio understands the meaning of certain property names and values. For example, this property in a project:

<OutputType>WinExe</OutputType>

will cause the "Windows Application" output type to be displayed in the project properties pane. The property value can be edited in the project properties pane and saved in the project file. If such a property is given an invalid value by hand-editing, Visual Studio will show a warning when the project is loaded and use a default value instead.

Visual Studio understands defaults for some properties. These properties will not be persisted into the project file unless they have non-default values.

Properties with arbitrary names are not displayed in Visual Studio. To modify arbitrary properties in Visual Studio, you must open the project file in the XML editor and edit them by hand.

Items defined in the project with arbitrary item types are by default displayed in the Solution Explorer under their project node. To hide an item from display, set the Visible meta-data to false. For example, this item will participate in the build process but not be displayed in the solution explorer:

            <ItemGroup>

                        <IntermediateFile Include="cache.temp">

                                    <Visible>false</Visible>

                        </IntermediateFile>

            </ItemGroup>

Items declared in files imported into the project are not displayed by default. Items created during the build process are never displayed in the Solution Explorer.

Conditions on Items and Properties

During build, all conditions are fully respected.

When determining property values to display, properties that Visual Studio considers configuration dependent are evaluated differently than properties it considers configuration independent. For properties it considers configuration dependent Visual Studio sets the Configuration and Platform properties appropriately and asks MSBuild to re-evaluate the project. For properties it considers configuration independent it is indeterminate how conditionals on them will be evaluated.

Conditional expressions on items are always ignored for the purposes of deciding whether the item should be displayed in the Solution Explorer.

Debugging

In order to find and launch the output assembly and attach the debugger to it Visual Studio needs at least the following properties to be correctly defined: "OutputPath", "AssemblyName", and "OutputType". The debugger will also fail to attach if the build process did not cause the compiler to generate a .pdb file.

Design-time target execution

Visual Studio attempts to execute targets with certain names when it loads a project. These targets include "Compile", "ResolveAssemblyReferences", "ResolveCOMReferences", "GetFrameworkPaths", and "CopyRunEnvironmentFiles". These are run so that the compiler can be initialized to provide intellisense, so that the debugger can be initialized, and so that references displayed in the Solution Explorer are resolved. If these targets are not present, the project will load and build correctly but the design-time experience in Visual Studio will not be fully functional.

Editing projects by hand

When it is necessary to edit an MSBuild project directly, or it is simply more convenient, right click on the project node in the Solution Explorer and choose "Unload Project". Then right click again on the project and choose "Edit Project". The project file will be opened in the XML editor and intellisense will be available. When you are done editing, save and close the project file, then right click on the project node again and choose "Reload Project".

Intellisense and validation

When using the XML editor, project file intellisense and validation is driven by the MSBuild schema files. These are installed by Visual Studio in the schema cache, which can be found beneath the <Visual Studio installation location>\Xml\Schemas folder.

Core MSBuild types are defined in Microsoft.Build.Core.xsd and common types used Visual Studio are defined in Microsoft.Build.CommonTypes.xsd. To customize the schemas so that you have intellisense and validation for your projects' item types, properties, and tasks, you can either edit Microsoft.Build.xsd, or create your own schema that includes the CommonTypes or Core schemas. If you create your own schema you will have to direct the XML editor to find it using the properties window for the XML file.

Editing loaded projects and targets files

Visual Studio caches the content of project files and files imported by project files. If you edit a loaded project file, Visual Studio will automatically prompt you to reload the project so that the changes take effect. However if you edit a file imported by a loaded project, there will be no reload prompt and you must unload and reload the project manually to make the changes take effect.

Output Groups

Several targets defined in Microsoft.Common.targets have names ending in "OutputGroups" or "OutputGroupDependencies". The names of these targets are well known to Visual Studio. Visual Studio calls these targets when it wishes to get particular lists of project outputs. For example, the "SatelliteDllsProjectOutputGroup" target's outputs are a list of all the satellite assemblies that building the project will produce. These output groups are used by features like publishing, deployment projects, and project to project references. Projects that do not define them will load and build in Visual Studio correctly, but some features may not work properly.

Reference Resolution

Reference Resolution is the name for the process of using the reference items stored in a project file to find the locations of actual assemblies. Visual Studio must trigger reference resolution in order to show detailed properties for each reference in the properties window. There are three kinds of references and here is how they are resolved:

Assembly References

The project system calls a target with the well-known name "ResolveAssemblyReferences". This target should produce items with the item type "ReferencePath". Each of these items should have an item spec containing the full path to the reference. The items should have all the meta-data from the input items passed through, in addition to the following new pieces of meta-data: "CopyLocal", indicating whether the assembly should be copied into the output folder, set to true or false; "OriginalItemSpec", containing the original item spec of the reference; and "ResolvedFrom", set to "{TargetFrameworkDirectory}" if it was resolved from the .NET Framework directory.

COM References

The project system calls a target with the well-known name "ResolveCOMReferences". This target should produce items with the item type "ComReferenceWrappers". Each of these items should have an item spec containing the full path to the interop assembly for the COM reference. The items should have all the meta-data from the input items passed through, in addition to the following new piece of meta-data: "CopyLocal", indicating whether the assembly should be copied into the output folder, set to true or false

Native References

The project system calls a target with the well-known name "ResolveNativeReferences". This target should produce items with the item type "NativeReferenceFile". The items should have all the meta-data from the input items passed through, in addition to the following new piece of meta-data: "OriginalItemSpec", containing the original item spec of the reference.
goodtimes
Just for the question of readability and mantainability of the proj file. It's much simpler this way. Until now, I haven't found problems with VS because of it.

Regards,
Gustavo Guerra
Mahesh D
I'm also very interested in a reply to this question. Currently I'm using the same setup as you do -- VS2005/MSBuild solution for editing and debugging, NAnt script for the real build. Consolidating everything into MSBuild would be nice, especially since MSBuild comes with the .NET Framework, unlike NAnt (or even Nmake).

I have a feeling we'll have to wait for some book entitled "Inside MSBuild" or the like for a definitive answer, though...
VS 2005 Integration with MSBuild

Answer this question