Hi,
My application is running on 65 PC.
After a few hours, this applications crash on 2-3 PC.
I get no exception in my log files.
I set the Application.ThreadException and AppDomain.CurrentDomain.UnhandledException exception event handlers.
The last one is triggered with 'System.NullReferenceException: Object reference not set to an instance of an object.' but there is no stack trace available.
I got no dump file, no popup window.
I only get the following entry in the event log :
Event Type: Error
Event Source: .NET Runtime 2.0 Error Reporting
Event Category: None
Event ID: 1000
Date: 13/05/2006
Time: 21:53:39
User: N/A
Computer: IPDIRECTOR51380
Description:
Faulting application ipdrouting.exe, version 2.0.24.19933, stamp 44645daa, faulting module unknown, version 0.0.0.0, stamp 00000000, debug 0, fault address 0x00902190.
For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.
Data:
0000: 41 00 70 00 70 00 6c 00 A.p.p.l.
0008: 69 00 63 00 61 00 74 00 i.c.a.t.
0010: 69 00 6f 00 6e 00 20 00 i.o.n. .
0018: 46 00 61 00 69 00 6c 00 F.a.i.l.
0020: 75 00 72 00 65 00 20 00 u.r.e. .
0028: 20 00 69 00 70 00 64 00 .i.p.d.
0030: 72 00 6f 00 75 00 74 00 r.o.u.t.
0038: 69 00 6e 00 67 00 2e 00 i.n.g...
0040: 65 00 78 00 65 00 20 00 e.x.e. .
0048: 32 00 2e 00 30 00 2e 00 2...0...
0050: 32 00 34 00 2e 00 31 00 2.4...1.
0058: 39 00 39 00 33 00 33 00 9.9.3.3.
0060: 20 00 34 00 34 00 36 00 .4.4.6.
0068: 34 00 35 00 64 00 61 00 4.5.d.a.
0070: 61 00 20 00 69 00 6e 00 a. .i.n.
0078: 20 00 75 00 6e 00 6b 00 .u.n.k.
0080: 6e 00 6f 00 77 00 6e 00 n.o.w.n.
0088: 20 00 30 00 2e 00 30 00 .0...0.
0090: 2e 00 30 00 2e 00 30 00 ..0...0.
0098: 20 00 30 00 30 00 30 00 .0.0.0.
00a0: 30 00 30 00 30 00 30 00 0.0.0.0.
00a8: 30 00 20 00 66 00 44 00 0. .f.D.
00b0: 65 00 62 00 75 00 67 00 e.b.u.g.
00b8: 20 00 30 00 20 00 61 00 .0. .a.
00c0: 74 00 20 00 6f 00 66 00 t. .o.f.
00c8: 66 00 73 00 65 00 74 00 f.s.e.t.
00d0: 20 00 30 00 30 00 39 00 .0.0.9.
00d8: 30 00 32 00 31 00 39 00 0.2.1.9.
00e0: 30 00 0d 00 0a 00 0.....
I have no clue where (or how) to search for the source of this problem !
Thanks in advance for any help.

.NET Runtime 2.0 Error Reporting
ümit önal
I didn't see how to use the critical region !
I guess my problem is in reading the Tcp command stream.
Clients for my application are sending about 1500 short messages/second.
I rely on the Tcp buffer and read (and handle commands) as fast as I can this stream.
What is the best way to read a Tcp stream
Should I try to read faster and put commands in a queue
Thanks in advance for your help.
Patrick Hampton
You can take the minidump back to a VS2005 machine and load it to get a view of the system when it went down. You'll want to make sure the binary files match the version that crashed but otherwise it should look similar to what it would look like if it had crashed while running the debugger.
The call stack you gave is all in unmanaged code. It appears that a thread is being started and the thread is getting a null reference. Are you perchance using ThreadStart or something to spawn a new thread Sort of sounds like a managed object is being freed before it is finished being used but that would be just a guess as to the problem. The callstack indicates that a new thread was started, the function to be called was passed down the stack until eventually it was called. Sometime during when it was called and it began execution it hit a null reference. You should examine the method that is being run on a separate thread to see if it is referencing a bad value. Since the callstack is bad at the top it is difficult to say what happened. You can examine the actual source code being used (or a close copy) by looking at Class.cpp in the Rotor v2.0 code. CallDescrWorker is the last method called before the call stack went into memory address land. This function ultimately calls the managed code that crashes.
Michael Taylor - 6/6/06
s4bt
I don't quite understand your conditional logic. You are checking to see if the exception object does not contain the text THREADABORTEXCEPTION. Why I would recommend that if you want detailed exception information then just log all exceptions to your custom log file. Also remember that unless you have the PDBs installed your callstack will probably be just a bunch of addresses. Other than that the code looks reasonable and should be called when any unhandled exception occurs (provided you hooked it up properly of course). You can verify it is working by throwing an unhandled exception at the top of your main program just to be sure the logging works. Also note however that you need to be careful here because if the app goes down because of lack of memory or a stack overflow then your handler will cause another unhandled exception and you won't get any logging. Therefore you may want to use the new critical region stuff in 2.0 to permit the logic to run even in these cases. Also note that you are assuming that you have write access to the local directory for your log which may not be true. You should probably do some checking for that case as well. There is nothing harder than trying to debug a problem with debugging code that is itself buggy.
Hope this helps,
Michael Taylor - 5/18/06
DJCheeky
As for the empty call stack remember that unless you are running a debug build or you copy the PDBs to the same directory as the binaries then you probably won't get much of anything as far as the call stack goes.
For release builds I believe the default is to generate the PDBs (debug settings in the project properties I believe) but you'll still need to copy all the PDBs to the same directory where your binary is running before the information would be available.
I'm not sure that is the problem you're having because there are a variety of ways the call stack could get stomped but it is pretty common. Unless you can hook the debugger up the only other option is to use the tracing facility to log calls into key methods until you can trace down the problem.
Michael Taylor - 5/30/06
Duque Vieira
My system administrator redirect the "Send" error report button to our server.
I got a cab file with a minidump in it.
I tried to set up debugging symbols and got the following call stack :
> 00902190()
mscorlib.ni.dll!793d7a7b()
[Frames below may be incorrect and/or missing, no symbols loaded for mscorlib.ni.dll]
mscorlib.ni.dll!793683dd()
mscorlib.ni.dll!793d7b5c()
mscorwks.dll!_CallDescrWorker@20() + 0x1f bytes
mscorwks.dll!_CallDescrWorkerWithHandler@24() + 0x9f bytes
mscorwks.dll!MethodDesc::CallDescr() + 0x115 bytes
mscorwks.dll!MethodDesc::CallTargetWorker() + 0x20 bytes
mscorwks.dll!MethodDescCallSite::Call() + 0x18 bytes
mscorwks.dll!ThreadNative::KickOffThread_Worker() + 0x156 bytes
mscorwks.dll!Thread::UserResumeThread() - 0x21728a bytes
mscorwks.dll!Thread::DoADCallBack() - 0x2162d2 bytes
mscorwks.dll!Thread::DoADCallBack() - 0x2163b0 bytes
mscorwks.dll!Thread::DoADCallBack() + 0x46a bytes
mscorwks.dll!ManagedThreadBase::KickOff() + 0x13 bytes
mscorwks.dll!ThreadNative::KickOffThread() + 0x255 bytes
mscorwks.dll!Thread::intermediateThreadProc() + 0x46 bytes
kernel32.dll!_BaseThreadStart@8() + 0x37 bytes
What is the problem
I really apreciate your help.
Thanks a lot.
june
I'm not sure about your stream problem because I don't do much with that stuff. As your original question was answered I believe your best option now is to repost this new question in the Networking forum. In that forum questions related to TCP in .NET are answered. You'll probably get a pretty quick response.
Nevertheless if you are dumping the call stack of the exception (through your log function) then it should identify at least where the exception is occurring so you can better diagnose the problem. If you provide that information along with your needs (1500 msgs/sec, etc.) then I'm sure someone will be able to help you out.
Thanks,
Michael Taylor - 5/30/06
PETER_WHITE123
I also use log4net for logging key methods but it doesn't help.
I still cannot understand how the dotnet runtime can log a runtime error in the logevent without any cause for this error.
Alex Araujo
So, I still don't know what cause this exception and this runtime error so I consider that my original question is not answered.
Though, you are right about the new question, I will repost it in the appropriate forum.
Thanks a lot for your help.
ReportUser
I started with the code from a colleague that did this test on THREADABORTEXCEPTION.
Then I tried quite a few things so it is not clean.
I wil remove this test.
I have no problem with code properly hooked or write access because I always get this System.NullReferenceException written in the log file.
I didn't know about critical region.
I will have a look.
I also fear a stack overflow or some exhausted resources.
Thanks a lot for your help.
Raj74
As a final resort I'd say drop a debugger on one of the problem machines and debug the problem. As for the debugger you could use either WinDbg (from the Debugging Tools for Windows) or the remote debugger for VS2005. This would eliminate the need to install VS2005 on the problem machine. Alternatively the ADPlus utility works well. You can read more about ADPlus and production debugging here http://msdn.microsoft.com/msdnmag/issues/05/07/Debugging/. Provided you have the PDBs then you won't need the source code on the problem machine to identify the function causing the problem. For WinDbg or the remote debugger you can just run the app. For ADPlus you'll need to configure it first.
Michael Taylor - 6/6/06
Martin Skala
The Event Log is good for finding out that something went wrong but it won't have debugging information available. Run Dr. Watson to see if a crash dump was generated (drwtsn32). If a log was generated it'll have the call stack information available from which you can debug the problem. Whether it generates a log or not is almost hit or miss in my opinion. It seems dependent upon system settings (such as Windows Error Reporting). Sometimes a log gets generated and sometimes it doesn't. Nevertheless if it does then, with the PDBs, the dump will contain everything you need. In fact you can take the dump file itself (if one was generated) and load it into VS2005 to get a verbatim picture of the process when it went down. Of course if you installed the debugger on the machine then Dr. Watson probably won't generate anything.
Michael Taylor - 5/31/06
Kevin Lindley
Here is the code :
private static void CurrentDomain_UnhandledException (object sender,
UnhandledExceptionEventArgs e)
{
try
{
string message = e.ExceptionObject.ToString();
if(message.ToUpper().IndexOf("THREADABORTEXCEPTION") == -1)
{
System.IO.StreamWriter sw = new System.IO.StreamWriter
("IpdRouting_Unhandled.LOG", true);
sw.WriteLine (System.DateTime.Now.ToShortDateString () + " @ " +
System.DateTime.Now.ToLongTimeString () + ">\t" + message);
System.Text.StringBuilder sb = new System.Text.StringBuilder (2048);
Exception currentExc = null;
for (currentExc = (Exception) e.ExceptionObject;
currentExc != null; currentExc = currentExc.InnerException)
{
sb.AppendFormat ("{0}\r\n{1}\r\n{2}\r\n",
currentExc.GetType ().FullName,
currentExc.Message,
currentExc.StackTrace);
}
sw.WriteLine (sb.ToString ());
sw.Flush ();
sw.Close ();
}
}
catch
{
//Application.Exit();
}
}
Thanks for your help.
Reza Shams
In order to get the stack trace and message from an exception you should be using the value from ToString() instead of Message. Message is simply the error message. However in order to get a reliable stack trace you'll need to be sure to put the PDBs in the same directory as the binaries. Otherwise the stack trace will probably just be a bunch of addresses. As an aside you could also use the stack trace information in the exception to build a nicely formatted string but this is not trivial even with the StackFrame and associated helper classes.
Michael Taylor - 5/16/06
Alexej
I inserted try-catch in all functions and still, 2 PC with this runtime error and the popup window with buttons "Send" + "Don't Send" error to Microsoft.
The program is still running but if the user click on the above buttons, the program ends.
I really have no clue how to solve this problem !