AccessViolationException when client is terminated

Hi,

We have just migrated our client/server application to .Net 2.0. After that we constant get an unhandled AccessViolationException in the server program when the client program is terminated!

The only stack trace I can get is:

at
System.Net.UnsafeNclNativeMethods.OSSOCK.WSAGetOverlappedResult(SafeCloseSocket
socketHandle, IntPtr overlapped, UInt32& bytesTransferred, Boolean wait,
IntPtr ignored)
   at
System.Net.Sockets.BaseOverlappedAsyncResult.CompletionPortCallback(UInt32
errorCode, UInt32 numBytes, NativeOverlapped* nativeOverlapped)
   at
System.Threading._IOCompletionCallback.PerformIOCompletionCallback(UInt32
errorCode, UInt32 numBytes, NativeOverlapped* pOVERLAP)


In the event log on the server the following is logged:

Faulting application MyModule.exe, version 1.1.1.30099, stamp 439eec16, faulting module nl_lsp.dll, version 0.0.0.0, stamp 406ae2c2, debug 0, fault address 0x000049fd.


The client and server are communication using .Net Remoting, and we do not have any other socket communication. Also, the server does not use any 3rd party code.

Everything worked fine before we migrated.

Thanks for any help you can provide...



Answer this question

AccessViolationException when client is terminated

  • Scott Snyder

    I have this exact problem too, and its 100% reproducable on some machine while its impossible on others. I tried the "config fix" as mentioned in this thread but I still can't catch the exception. My server is currently a simple console app (I'm still prototyping my code) and here's the very simple code:

    Server.Publish(); // <- this method publishes my server
    ConsoleKeyInfo key;
    do
    {
    try
    {
    key = Console.ReadKey(); // <- server should be here when client terminates
    }
    catch (Exception error)
    {

    // I never get here!
    Console.WriteLine(error.Message);
    }
    } while (key != null && key.KeyChar != 'q' && key.KeyChar != 'Q');
    Console.WriteLine();

    While the former AVE is now a NullReferenceException, it is not catched whit the above code. Is there some other way of doing that


  • johnny wisconsin

    moggel wrote:
    The "culprit" turned out to be my anti virus scanner (NOD32). This post - http://lab.msdn.microsoft.com/productfeedback/viewfeedback.aspx feedbackid=d86baf6d-9dcb-408f-8fd8-efebc237edc8 - got me to try and exclude the server exe from being scanned. Having done so the server no longer crashes.

    I now believe it must be a bug in the .NET 2.0 remoting framework that doesn't handle exceptions that may be thrown in the unmanaged socket layer of Windows. Reading the stack trace, it seems a callback from an unmanaged, socket-related, dll back to managed code goes wrong.

    I'm filing a bug report on this one...

    Pretty sure I said that earlier...let's look and see

    ivolved_mike_brown wrote:

    Doing a little more scouring, it appears the problem is one of native to managed code re-entrancy. The Managed code calls native code that callsback into ManagedCode. There is a group of tools called Managed Debug Assistants, that will give you more information regarding what exactly is going wrong with the code. You can learn more about them here

    Yep that's me. But thanks for filing the bug report. I'm sometimes too lazy to do that type of stuff



  • js-mueller

    I think your assumption (that some firewall or antivirus might be in the way) is true, since the AVE only threw on one of my two machines which has McAfee installed (the other one is merely protected by Windows Firewall, and it's always able to catch the "Remote host is forcibly shut down" SocketException.)


  • Amadeus156553

    Doing a little more scouring, it appears the problem is one of native to managed code re-entrancy. The Managed code calls native code that callsback into ManagedCode. There is a group of tools called Managed Debug Assistants, that will give you more information regarding what exactly is going wrong with the code. You can learn more about them here



  • com100h

    Any new thoughts on this I hit exactly this problem, straight away. I can't find a way to terminate a client without crashing the server. I get the same exception as desribed here.

    The exception does NOT happen if I let client and server sit idle for several minutes.

    I can look at my LSPs as suggested, but I have nothing foreign installed there (that I know of).



  • ooodi

    Can everyone who is experiencing this particular problem (AccessViolationException on .NET Remoting 2.0 server when the client is terminated) please contact me directly at eugeneos@microsoft.com

    The Remoting team is very interested in getting to the root of this problem and we need some more information from you.

    Thanks!



  • gs2006

    jimh12345 wrote:
    I should have checked my event log. It shows ciphire-lsp.dll as the faulting LSP.

    Ciphire is an email encryption service which I do NOT want to give up. Our customers may also have LSPs that they need. Surely there is some way to gracefully terminate a client in .NET remoting...


    See my response higher up. You can configure the .Net runtime not to throw the AVE.



  • wx

    The "culprit" turned out to be my anti virus scanner (NOD32). This post - http://lab.msdn.microsoft.com/productfeedback/viewfeedback.aspx feedbackid=d86baf6d-9dcb-408f-8fd8-efebc237edc8 - got me to try and exclude the server exe from being scanned. Having done so the server no longer crashes.

    I now believe it must be a bug in the .NET 2.0 remoting framework that doesn't handle exceptions that may be thrown in the unmanaged socket layer of Windows. Reading the stack trace, it seems a callback from an unmanaged, socket-related, dll back to managed code goes wrong.

    I'm filing a bug report on this one...


  • Nilesh Rade

     Nicolai G. Andersen wrote:

    Thank you for the reply.

    I found the root of the problem (well at least for some of it). The error logged in the event log was on the nl_lsp.dll. This is a dll used by a program "Netlimiter" that I had installed for some bandwidth tests. After un-installing that program everything worked fine. 
    However, I have now found out that other people have seen similar problems where the causes have been different firewall or virus protection software. It might be the case that these programs are doing something illegal, however I can't see why .Net must crash like that. Also, I did not have a problem before I migrated.
    It does not make me sleep very well knowing that the behaviour of my program has changed that dramatically after a "simple" upgrade to the new framework version, and that there exist several "normal" programs out there that will cause my app to crash.

    It definitely smells like a bug to me. Framework code should not be throwing an AccessViolationException. However the current workaround is to make that change to the configuration file and catch any NullPointerException in your code so it won't crash.



  • zhili

    Thank you for the reply.

    I found the root of the problem (well at least for some of it). The error logged in the event log was on the nl_lsp.dll. This is a dll used by a program "Netlimiter" that I had installed for some bandwidth tests. After un-installing that program everything worked fine. 
    However, I have now found out that other people have seen similar problems where the causes have been different firewall or virus protection software. It might be the case that these programs are doing something illegal, however I can't see why .Net must crash like that. Also, I did not have a problem before I migrated.
    It does not make me sleep very well knowing that the behaviour of my program has changed that dramatically after a "simple" upgrade to the new framework version, and that there exist several "normal" programs out there that will cause my app to crash.


  • wjxia

    Hi there,

    I also encountered this uncatchable exception lately, when I was trying out zero-sending-buffer socket communication. The trigger is also that one client terminates ungracefully (I pressed Ctrl+C). The exception is uncatchable because it's run in a thread not maintained by our program--it's a low-level callback on completion of some IO things; I got this info from the stack trace, which is exactly the same as the one you showed above.

    I then came up with the idea of forbidding the IL from calling the completion callback by setting Socket.UseOnlyOverlappedIO to true. According to the reference, this disables the completion port bla bla bla, although I have no idea at all what it does. But it helped--instead of uncatchable AccessViolationException, I caught an "unknown error (0xc00000b5)" (the hex code will change) on EndSend/EndReceive. Although this says nothing, it still can be caught and the program can resume, instead of letting Windows pop up a "Program terminated unexpectedly. Send error report to Microsoft " thing.

    But this can be done only if you have access to the socket object, and I believe under the remoting environment it's sealed underneath. Maybe you can find a way to handle the remoting by code and manual socket communication instead of automatically


  • gloira

    I found this:

    http://msdn.microsoft.com/netframework/programming/breakingchanges/runtime/clr.aspx

    Do a find within that page for AccessViolationException. Apparently the Remoting runtime does not handle ungraceful client termination well.

    However I think it goes a little deeper than that. Is your server in the process of sending/receiving data when the client terminates (e.g. does it have a handle back to the client). If so, try putting a finalize method into your server object that tells it to release that handle.

    Right now I'm in the world of guesstimating because I don't know precisely what the server is doing when the exception is called. Try debugging through the server when you exit the client and post exactly what is happening when the AV is thrown.

    Alternatively, you can look here to find more about the AVE and how to suppress it from the configuration file. This seems like it is definitely a bug because the Framework shouldn't be throwing AVEs from what I read.

     



  • Golgot13

    Hi,

    do you have found a solution. I have exactly the same issue like you.
    I also don't know how I could catch of either exception.

    Thanks,
    Stefan

  • ranganathanmca

    I should have checked my event log. It shows ciphire-lsp.dll as the faulting LSP.

    Ciphire is an email encryption service which I do NOT want to give up. Our customers may also have LSPs that they need. Surely there is some way to gracefully terminate a client in .NET remoting...



  • AccessViolationException when client is terminated