I've found several references in forum and blogs to DeleteTeamProject.exe or TFSDeleteProject and some mention of it being in C:\Program Files\Microsoft Visual Studio 2005 Team Foundation Server\Tools.
Such a program doesn't appear to be in my installation. Was it removed for the December CTP

Delete Team Project missing in CTP?
Mario Esposito
Hi Andrew,
I see "TFSDeleteProject.exe" in ...\Visual Studio 2005 Team Suite\Common7\IDE\PrivateAssemblies. Do you have such a folder
JimmyB
I found it there moments ago. The actual problem I had was that I was trying from the standard cmd prompt instead of from the Visual Studio 2005 Command Prompt, where there is a better PATH setting and some number of environment variables might also be helping.
To use the delete, go into the Visual Studio 2005 Command Prompt ( under Start -> Programs -> Microsoft Visual Studio 2005 -> Visual Studio Tools).
Then finding where it was was a simple "where tfsdeleteproject":