PAGE_EXECUTE_READWRITE and the CLR -- debugging issues

PAGE_EXECUTE_READWRITE and the CLR -- debugging issues

• Ruud Baars

  After some investigation, it appears that the window shows that error message ("The specified address cannot be displayed...") when the debugger is in a managed frame in the callstack (use the [Native to Managed Transition] and [Managed to Native Transition] thunks in the callstack window to identify what type of frame are you in).

  As a workaround, move your debugger's context to a native frame (by double-clicking a native function in the callstack). Now you should be able to set the Address in the Disassembly window to any address in memory and then specify the breakpoints or tracepoints you want for that code.

  -- Marian Luparu
  Visual C++ IDE

  
  

• ShodanTech

  Hi --

  Thank you so much for responding, but unfortunately that doesn't seem to have done the trick.

  My settings in the Debugging options screen are as follows:

  Command : $(TargetPath)

  CommandArguments: (blank)

  Working Directory : (blank)

  Attach: No

  DebuggerType: Mixed

  Environment : (blank)

  Merge Environment: Yes

  SQL Debugging: No

  Is there anything else you can think of that might be affecting this

  FYI,

  Here's the call that allocates the memory:

  m_Image = (BYTE*)VirtualAlloc(NULL,1024*1024,MEM_COMMIT,PAGE_EXECUTE_READWRITE);

  The Intel instructions written to this memory are definitely being executed, because I see the strings that are ultimately being written to the OutputWindowPane I've added. It's just the disassembly window that won't cooperate.

  Any other suggestions would be immensely appreciated. If I can't get past this it has serious implications for our ability to integrate with Visual Studio!!!

  Regards,

  Mike

  
  

• andylongo

  Hi Marian

  Thank you so much, that did the trick. I am still a bit perplexed about one thing, however. My C++ program doesn't do very much except write an assembly language image to the memory it has allocated and then execute it. Thus there weren't very many calls on the call stack to choose from. I noticed that the topmost one was something called mainCRTStartup, which is reported as a C function. Then there's a thunk to my app's main function, which is now running in a managed frame. I have a single class that initializes and executes the assembly code and what is perplexing to me is that the class's constructor runs in a managed stack frame, but the call to Execute (made from within the constructor) thunks back to native code. Is the explanation something like the following: that the constructor is invoked by managed code so it runs in a managed frame, but the class itself is unmanaged, so anything done from within in runs on a native frame Is that what's happening Just curious.

  Now I just have to figure out the best way to synchronize my calls into the DTE, which are made not from an addin, but from the app being executed (clearly from another thread, if the number of "server busy" messages I get is any indication). Is the best practice for this documented anywhere that you know about I was thinking that maybe I need to run a tiny addin that exposes the entry points to the DTE. Would that do it This is obviously something I can research, but if you happen to have any suggestions they would be greatly appreciated.

  Thanks again for your help!

  Mike

  
  

• Peter Nyberg

  "Is thhis intended behavior and is there something I can do to re-enable debugging of my generated code "

  It looks like you are in native mode debugging while trying to debug managed code or viceversa. Try setting the debugger to "Mixed" from the Property Pages / Debugging / Debugger Type. Stepping in disassembly should work then.

  "jumped from around 2MB to around 16+MB just by compiling with the /CLR flag. Is that normal Or is there stuff I can do to reduce this footprint "

  When compiling with /clr you bring the world of .NET into your application. And usually .NET applications have a larger footprint than native applications (try to create a "Hello World" application in C#/VB and you will notice the same memory requirements).

  An aesthetic "solution" would be to call:

  SetProcessWorkingSet( GetCurrentProcess(), -1, -1);

  That will drop the mem usage in TaskMan by freeing the memory .NET runtime reserved at start-up and is not used yet. But I don't think this will actually improve your performance in the long run so you should check out these articles from MSDN:

  Writing Faster Managed Code: Know What Things Cost

  Writing High-Performance Managed Applications : A Primer

  Two things to avoid for better memory usage

  -- Marian Luparu
  Visual C++ IDE
  

  
  

• Mike Williams

  "I have a single class that initializes and executes the assembly code and what is perplexing to me is that the class's constructor runs in a managed stack frame, but the call to Execute (made from within the constructor) thunks back to native code."

  Probably the constructor is implemented in a .cpp file that is compiled with /clr and the Execute function in a .cpp file that is not. It does not matter who makes the call, but how the function gets compiled.

  The thing with IJW is that you have to be careful how many times you do the flip between managed and native, and avoid duble-thunking (going from managed to native and then back to managed without doing anything). Then again, reading your program description, I do not know why you need /clr in your app in the first place.

  "Now I just have to figure out the best way to synchronize my calls into the DTE"

  You should ask these questions in the Visual Studio Extensibility forum. I think that you cannot access DTE objects from outside the devenv.exe process but you should double check that.

  -- Marian Luparu
  Visual C++ IDE

  
  
• PAGE_EXECUTE_READWRITE and the CLR -- debugging issues