Suprising performance increase

Hello,

I recently stumbled across a huge performance difference involving writing out a large hash table to a file.  The hashtable is holding a very simple object called Token, that holds a string value called token and a couple counters.  ToString() is overriden for the object to return:

tokenstring [space] counter1 [space] counter2

If I iterate through the values of the hashtable and write it out to a file, it takes around 1 minute and 40 seconds to complete.

Seems a little slow.  Here is the weird part.  If I iterate through the values of the hashtable and add them to an arraylist and then iterate through the arraylist to write out the values, it takes between 10 and 12 seconds.

The hash table is large, as the file that is written out is 39 Mb where each line is around 18 characters, where each line represents one hash table entry.

Below is some of my test code to clarify the situation if my explanation above has anyone confused.  WriteTokensSlow takes about 1 minutes and 40 seconds.  WriteTokensFast takes between 10 and 12 seconds.

private void WriteTokensSlow()
{
  WriteCollection(tokensHT.Values);
}

private void WriteTokensFast()
{
  ArrayList tokenList = new ArrayList(tokensHT.Values);
  WriteCollection(tokenList);
}

private void WriteCollection(ICollection collection)
{
  using (StreamWriter sw = new StreamWriter(savePath,false))
  {
     foreach (Token tokenItem in collection)
       sw.WriteLine(tokenItem.ToString());
  }
}

Can anyone explain why this is happening   Thanks.

Answer this question

Suprising performance increase

JimDan
Actually, if you replace the code that passes the hashtable into the arraylist with one that iterates through that hash table and adds the values to the array list, it will still perform fast. So it isn't iterating through the hashtable that takes so long. That's why I am so confused. I have no idea what is causing slow function to be so slow.
rbanke
really wierd.

The only difference I can see then is that your iterating a ICollection, maybe the casting is slower, I don't know

Hey, where's Ken Getz when you need him!
vjnfjvbnhjbvgfb
Sorry I misread your post, that Is wierd!

// this is what has to be expensive , hmmmm
foreach (Token tokenItem in collection)

I wonder if you didn't do anything but iterate if you would get the same results.

that would mean that iterating items in a hash table is very very slow, this is good to know :)
mina_mina
Serialization is slow no matter what, I'm not sure that there is anything you can do.

Nata1 used to serialize and deserilize the main BST to disc, and it would take about 2 minutes!

Then I serialized myself to xml and got the number down. There should be some components someday that will make this go faster, and hopefully this is improved in 2.0
RayCh
Hi Nata1,

I'm not using serialization.

I am just looking for anyone who might know why loading the objects from the hashtable into an arraylist before writing the data to a file would be such a huge performance increase verses writing the objects out from the hashtable.

The line that causes the huge performance increase is:

ArrayList tokenList = new ArrayList(tokensHT.Values);

Comment that line out of WriteTokensFast() and it takes around 10x longer.
Suprising performance increase

Answer this question