Datasets aren't finalized after leaving the scope

Hot Topic

2 Connections? Why?
Where should a config file be stored?
What does the "local-name" mean?
Common Language Runtime Debugging Services
Using ODP.Net with an XML Service
IE hosted 2.0 object - Configuration system failed to initialize
Slow load of projects in VS.net and how to make StringBuilder like for arrays marshalling
cannot convert from System.Data.IDbTransaction to System.Data.Common.DbTransaction
Code not working on German OS
CLR Profiler Question

.NET Development

ASP.net and CDONTS???
Data changes not being saved to the .mdf file
String or binary data would be truncated
.NET High Speed Timers - Precision/Accuracy
Calling RaiseEvent over Reflection?
Can a webservice exist with a WSE3.0 method and a non WSE3.0 method
xml data into SQL Server table
Using in Statement with DataTable.Select in ADO.Net
Reusable classes stored outside of the current solution.
Data Row Nulls

Datasets aren't finalized after leaving the scope

Hi,

we discovered a strange behaviour concerning garbage collection of datasets.

Because our application needs a tremendous amount of memory and constantly allocates more and more of it we are counting the instances of our objects in memory. Thus we able to determinate if an object is finalized by the garbage collection.

Basically most of our objects contain these code snippets in the "new" and "finalize" methods:


Public Sub New()
   MyBase.New()
   TestClass.SharedCounter += 1      'Shared variable of integer
End Sub

Protected Overrides Sub Finalize()
   TestClass.SharedCounter -= 1      'Shared variable of integer
   MyBase.Finalize()
End Sub
 



This works fine for many objects like buttons, forms, menustrips etc. But unfortunalty datasets don't seem to be finalized at all as the counter never decreases. Because our application instantiates a whole bunch of short living huge datasets this could be one reason for our memory leak.

A way to reproduce this behaviour is to create a new class that inherits Dataset and contains the above mentioned lines. This class has to be instantiated like this:


Public Sub Test()
Dim _dataset as CountedDataSet = New CountedDataset
End Sub
 


After each call of this sub the counter is increased but never decreased even if you manually call GC.Collect.

Does anybody know how to get rid of dataset instance

Thanks a lot!

Answer this question

Datasets aren't finalized after leaving the scope

  • marcin.walus

    Yes - we also tried to call the dispose method. But regarding to the following posting it doesn't have any effect at all:

    3. The Dispose method on a Dataset as well as a DataTable does nothing and it is not necessary to call them. The method has been included from the perspective of future design changes or for the purpose of derived classes which may use some unmanaged resource and need the dispose.


    (See http://forums.microsoft.com/msdn/ShowPost.aspx PostID=65110)

    Could you please give us an example of your idea  We just know the "AddressOf" operator for event handling. To which dataset do you want to compare to after the object should have been destroyed (=after left the scope).

    We think that there might be any internal structures or lists that reference to the dataset (or DataGridView as you can see the same behaviour with this control). Because of that the garbage collection doesn't recognize them as "not used anymore".

  • willstewca

    Yeah your right AddressOf can only accept method pointers.
    I was assuming you could use AddressOf like a pointer to an object in C#

    Im stumped at the mo, i might be able to provide some answers later when im at home with VS.net in front of me a my trusty reference guide.

  • MarlaF

    Hi TAS,

    I would strongly recommend against using finalizers for object lifetime bookkeeping. Finalization is not deterministic, nor guaranteed, so your object may be dead (no more live references to it), but not finalized.  Also, finalization incurs a perf hit, since objects that implement finalizers need to be promoted a generation before their finalizers are run, and then collected.  This includes all objects your finalizable objects refer to as well, and so on down the object graph.

    A better way to determine if objects are still alive would be to use the debugger. 

    Is there a reason why you're monitoring object lifetimes like this   The GC should collect your objects when they no longer have references to them.

     

    Lee_Dale,

    I think you're confusing Dispose() with finalizers.  The GC does not call Dispose, rather it spawns a finalization thread that calls the finalizers of objects at some point after they are considered dead.

    Also, you cannot view the address of an object because after each garbage collection, the object's has probably changed (unless the object is pinned, which is discouraged for perf reasons).

     

    Hope that helps

    -Chris [MS]

  • mnTeddy

    Hi Chris,
    it sounds that I have a similar problem. I believed that your suggestion is my solution too. But it isn't. I have a memory leak with datasets in conjunction with Compact Framework.
    To verify the problem I have made a small program listed below. But I get an error after the fourth call (the german message is: "Fehler bei systemeigener Ausnahme"
    Ausnahmecode: 0xc0000005).

    Public Class CountedDataSet
        Inherits DataSet
        Friend Shared Counter As Integer = 0
     
        Public Sub New()
            MyBase.New()
            GC.ReRegisterForFinalize(Me)
            Me.Counter += 1
        End Sub
     
        Protected Overrides Sub Finalize()
            Me.Counter -= 1
            MyBase.Finalize()
        End Sub
    End Class


    Is there anything wrong with this code in combination to your suggestion Or is it possible that there is a problem with compact framework
    Another question is: could you observe memory leaks with datasets in Compact Framework And if so, do you have a solution

    Thanks a lot.

    Markus

  • snorlaxnet

    We just discovered that this behavior can also be reproduced with a DataGridView. Try to instantiate a DataGridView and leave the scope (as shown in the sub above). You will see that the counter doesn't decrease as it does with other objects.

    If you place a DataGridView onto a form, the form will also not be finalized. There seems to be a reference from the DataGridView to its parent form, which prevents the form of beeing finalized.

    Because we use a lot of DataGridViews on our forms this should be a second reason for our memory leak.

  • dpuccio

    Hi TAS

    Thanks for the thorough description of your situation. 

    In the DataSet constructor there is a call to GC.SuppressFinalize().  So in your DataSet subclass, you must add a call to GC.ReRegisterForFinalize(this) in your constructor to override their SuppressFinalize call.

    (Why DataSet does this, I don't know, but you can ask in the Data Access and Storage forum: http://forums.microsoft.com/msdn/ShowForum.aspx ForumID=45.)

    After removing your DataSet subclass from the panels, you can add calls to GC.Collect() and GC.WaitForPendingFinalizers() (in DEBUG), which will run the finalizers for all dead objects before continuing.

    If all that doesn't work, there must be a live reference (or GCHandle) keeping your DataSets alive.  Unfortunately there is no way with managed code to determine what objects are keeping others alive, so you would have to use the SOS debugger extension.

    Hope that helps

    -Chris [MS]

  • billhu

    We used a blank dataset for this test. We just added the counter functionality:


    Public Class CountedDataSet
       Inherits DataSet

       Public     Sub New()
          MyBase.New()
          Class.SharedCounter += 1
       End Sub

       Protected     Overrides Sub Finalize()
          Form_DatasetTest.Counter -= 1
          MyBase.Finalize()
       End Sub
    End     Class
     


    Therefore there shouldn't by any objects or references that block the garbage collection of the dataset. AFAIK a finalizer is always called by the garbage collection if there exists one.

    See http://msdn.microsoft.com/library/default.asp url=/library/en-us/vbcn7/html/vaconfreeingreferencestoobjects.asp

    Finalize

    Another method supported by some classes, Finalize, runs automatically when an object is released and can be used to perform other cleanup tasks. The Finalize method is similar to the Class_Terminate() method used in previous versions of Microsoft Visual Basic. Unlike the Dispose method, the CLR automatically calls the Finalize method sometime after an object is no longer needed.


    All other components - like forms, buttons, textboxes, panels, bindingsources, etc. - are finalized very well and counted back.


  • circusfire

    Hi Chris,

    thank you for your help. As I described above we basically detected, that our application needs a tremendous amount of memory and constantly allocates more and more of it.

    Therefore we looked for a possibilty to find out, what causes this behaviour. So we created a class that counts the instances of classes. Each countable class includes a constructor and finalizer the increase and decreases the instances count. Because we were aware of performance issues, we added this code fragments in "IF #DEBUG" conditions so it won't be compiled into release versions.

    We read about the indeterministic behaviour of finalizers that is caused by objects that stop the garbage collector to finalize them in their dispose methods. We found out that this is the reason why the finalizer of forms hasn't been called. To avoid this indeterminism we run GC.ReRegisterForFinalize during dispose (also for #DEBUG compilation).

    After insertion of this command our object counter works fine even for shown forms as well as for all other (visual) components - except DataSets and DataGridViews! We tried to insert the ReRegister command into this objects too, but it still doesn't work.

    We also discovered that the non-finalization of an object affects its parents. If you add a datagridview to a form, that form isn't finalized anymore.

    From the starting point this behaviour would exactly explain the memory leak of our application. We created a framework that is able to create endless forms (like you can see in Access). An endless form contains panels (user controls) and each panel has a set of textboxes, comboboxes, checkboxes, etc. To be able to do the data bindings using the visual designer, we added an instance of our main dataset to each panel. This instance is disposed during the construction of the panel because the datasource property of the main bindingsource is set to the central dataset that contains the data. It exists just for designtime purpose.

    As you can imagine the application will create a lot of dataset instances while showing an endlos form. Because the dataset is fairly large (about 5MB source code file) it's instance need a lot of memory.

    Although the dispose method doesn't help to reduce the memory usage while showing the endless form (as we know now), it should be reduced after closing the form (=leaving the scope). Or at least the application shouldn't allocate more memory while showing the form twice (because of the windows memory management). But this is what happens - each time the form is shown, the application needs about 20MB more memory. And this behaviour would lead to massive memory problems in realtime usage.

    In order to increase the overall performance we removed the dataset instances from the panels and added the data bindings manually. But there are still instances of datasets and datagridviews that should be finalized after disposing a form.

    We could think of internal lists for management purpose, that contains references to datasets and datagridviews. They are not removed after the scope in the main application has been left so the garbage collector will never remove these objects. I cannot access them during debug because my application doesn't have any references anymore...

    I hope this will help to understand the basic problem and maybe someone has further ideas

    Thanks a lot!

  • Carlton Lane - MSFT

    Declare a global variable to hold the memory address of the dataset, then when the dataset is created say globalVariable = AddressOf (mydataset).

    This would store a pointer to the memory address of teh dataset.

    Then when the dataset is destroyed you could do a compare to a dataset object to see of the memory address contains a dataset object.

    e.g.

    If globalVariable Is DataSet Then
       MessageBox.Show('Is DataSet')
    Else
       MessageBox.Show('Is Not DataSet')
    End If

    if the memory address is a dataset object then you can be pretty sure the object is not getting destroyed. But im failry sure the DataSet will not exist anymore.

    I have no way to test this code at the moment so im assuming you can do this.



  • Ralph_Ai2x

    Thanks for your ideas but we don't see any "addressOf" command. The only addressOf operator is used to add handlers to events of controls. Its syntax is "AddressOf(<name of a method>)".

    By the way objects are frequently moved in memory during garbage collection. So it propably wouldn't be helpful to compare memory addresses even if there is a command to get them.

    Besides we found another object that is not proberply finalized: the form object. As long as you don't call "show" it get's finalized just as you would expect it after leaving the scope. But if you call "show" and immediatly "close" it stays in memory until the application terminates.

    We could give you a sample project to reproduce it.

  • Shawn Weitzel

    Hi Markus

     The error message you provided (roughly translated in English as "Error with system exception") isn't descriptive enough for me to figure out where the problem lies.  If there is an exception being thrown, it would be helpful to find the type and a stacktrace.

    The Compact Framework uses different classes and a different garbage collector than the full framework, which is a little out of my area of expertise :)

    I suggest you ask this question and provide the exception information on the Compact Framework (http://forums.microsoft.com/msdn/ShowForum.aspx ForumID=33), and someone there should be able to help you out.

    -Chris

  • Amit Barde

    have you tried calling the dispose method

    Can't think of any reason why the GC wouldnt free up memory for a dataset.

    try setting a variable to AddressOf your dataset then after the object is destroyed compare the AddressOf variable to a dataset and see if the actual memory space is freed up

    Just a thought

  • rezna

    It would suggest that your dataset object is creating resources that are not being finalized, so in turn the finalize method for the dataset never gets called.

    If your dataset references other objects make sure you call the finalize method of them objects when your are finished with them

    Although the GC can monitor the objects lifecycle it doesnt know how to clean up internal resources created by that object.

    Also if an object is unreachable the GC calls the dispose method of the object which skips over the finalization method and thus you counter variable will not get decremented.

    It may be that the memory is actually being freed up but your counter is not decremented because the finalize method isnt being called.

  • John Neighbors

    Hi Chris!

    Now it works! We added ReRegisterForFinalize to the overridden dispose method and had no success. But after inserting this command into the constructor our counter decreases as expected! (We don't understand what makes difference but sometimes it seems like you don't have to understand everything...)

    Thanks a lot for your ideas!
  • Datasets aren't finalized after leaving the scope
    • Answer this question