Is there a way to make the build server queue build requests instead of running them simultaneously We have had several problems with builds hanging during initialization, hanging during building, or completing the build process but failing to drop to the network when two builds are competing.
I have considered writing a task that will wait on a mutex at the start of every build, but thought maybe I had better ask if MSFT had already planned for this and had some built in setting I could tweak.
Thanks in advance.
MikeH
Is it possible to queue Team Builds?
Misha Shneerson - MSFT
Hi Gautam.
The larger project has a total of 41 .sln and 88 .csproj files. Everything is done in C#.
Each solution has 1-5 projects, with most having 2 projects.
I dont have time to draw the dependency tree right now, but yes there are several intertependencies. A few of the projects are common dependecies for all other projects, but a vast majority is projects that depend on several projects in several other solutions.
Obviously there are no circular project depencies, but there are several cases of solution interdepencies such as Solution1Project1 depends on Solution2Project1 and Solution2Project2 depends on Solution1Project2. -- This may not be the best, but this was all set into motion long before I worked here.
biswebdotws
Also, why is there a restriction on one team build per team project per build server
It seems odd that I can run multiple builds together, but only if they're from different Team Projects
fradi
We have had this problem both with Beta 3 Refresh, and to a lesser degree with the RC version. I will say it does seem to happen less frequently since the upgrade to RC, but it is still happening.
Anu,
I have a nightly build process, but we also potentially build a few times throughout the day. There are 5 different Team Projects using the single build machine. One of these is a substantially sized project that takes several minutes to build. The other projects are relatively small and build within a minute or two. The problem seems to occur when the large project is one of the two being built -- perhaps it is a problem with memory
Also, yes killing the build process does work instead of a reboot.
maria paramita
What is the structure of the bigger one By this I mean number of .sln, number of *.*proj within each .sln, language (i.e. C#, C++ etc) and dependency across these *.*proj files. If you can give the dependency tree, it will be great.
Thanks,
Gautam
LoekyH
Hey Anu,
Thanks for responding to me.
I am not quite sure what the deal is, but I would guess that 5-10% of every build we start hangs in the initialization (Initialization or Getting sources) phase. Maybe another 5-10% hang at some other phase. Most of these I have been able to take a quick look at the build server and notice there were 2 team builds running at the same time. This is why I was trying to queue the builds -- to see if this is the problem.
We have also frequently noticed that once one of these builds hangs, all attempts to start another build will also hang until we reboot the build server. This happens maybe 30% of the time when a hang is experienced.
Also, once a build hangs it never updates the build steps. You can look at the build screen 3 days later and it still says the hung build is 'Getting sources'. It is a minor thing, but there have been many comments here that it should be able to detect that thread died and update the screen as such. (My boss likes to comment 'You know how many phone calls I would get if I left a screen stating it was updating for 3 days ')
For now, I may try to create some mutex to see if this helps or fixes our problems.
I do think the RC is a big step up from the betas, and want to thank the TFS folks for the time they take to really help out on these forums.
--MikeH
Ravish Subramanya
On what build are you experiencing the hang - Beta 3 Refresh or RC After Beta 3 Refresh we have fixed few issues in this area. With RC bits we have tested with 10-50 simultaneous builds.
To answer the other question - you can have one build per team project per build machine. However, you can share build machine across team projects or have multiple builds for the same team project across build machine.
Thanks.
kolbe144
We dont have this feature in this version, Mike. However, we will look into adding this feature into future versions. Thanks for the suggestion.
BTW, why are your builds hanging during initialization and building Is this a known issue with the built code or do you suspect a Team Build issue that you are experiencing
Joe Jones
Thanks Mike for the details. We will try to see if can repro it.
Meanwhile, the next time it hangs, I want you to save the BuildLog.txt file. Since the build is not complete, the file will there only under build directory. We may need that so see what is going on.
As for the dependency tree, for one of the successful builds of the above project, go to the build report and send me the picture of Build Steps... that will help us build the tree.
You can mail me at gautamg at the rate microsoft.com. If you already have one such log file, you can send that too.
Thanks,
Gautam
GDocherty
Hi Mike - I wanted to check 2 things with you:
1. You mention there are 2 builds on the machine. Do you kick off builds based on some event or at regular intervals Are the ahngs experienced only during concurrent builds
2. Have you tried killing the build processes and kicking off a new build instead of the build server reboot
Thanks for reporting this. You could also mail me at anutthar at microsoft.com if you want to continue this discussion over mail. Thanks!