Keep getting Virtual Memory errors

Keep getting Virtual Memory errors

• XE Paul

  What's CPU usage like, is it at 100% If so, who's hogging it In Task Manger, under Processes, add counters Peak Mem Usage, VM Size, Handles, Threads, etc. or whatever else that can point to see who's using the memory.

  You may also want to double-check that your indexes and replication filters are optimized properly, make sure you have indexes on all the join filter columns.

  
  

• Dan Essin

  Hi, John,

  I have reproduced your issue on our lab machines. Dev are working on it.

  Some information to share with you...

  Here is what I did:

  1. configure merge publication with pull subscription one a table with only primary key and image columns
  2. run initial snapshot, and run merge agent through web sync
  3. insert into subscriber database with 100 rows and each row has image data 500KB

  You can see replmerge.exe process keeps increasing memory up to 868M.

  Then, the process is terminated by OS with log error:

  The merge process could not allocate memory for an operation; your system may be running low on virtual memory. Restart the Merge Agent.

  At the same time, CPU occupation looks fine, replmerg.exe use up to 10% at most.

  The lab machine I am using has 3GHz CPU and 1G RAM.

  BTW, there are some other data, in case they are helpful.

  When every row with 500KB image data, uploading 20 rows works fine.

  When every row with 1GB image data, the maximum rows can be uploaded is 12. More than 12 rows will cause merge agent failed with error “incorrect format”, it may be because it hit the maximum size of each message can reach.

  
  
  

• Michellos

  Yunjing,

  Thanks for the update. I am assuming this isn't supposed to happen correct Any idea of when a workaround or fix could be possible

  John

  
  

• Juan M. Aguirregabiria

  Greg,

  Initially when I first started the post we didn't have the @stream_blob_columns set so I'm assuming it defaults to 0. I then set it to 1 based upon your previous post. Basically we've had it set both ways and still get the memory error. Other thoughts

  
  

• kinetoplast2000

  Greg,

  Yes we're getting this message on the subscriber. I am really not sure where to start It seems that the RAM is getting maxed out and then the page file grows to it's max so we then get the virtual memory error. I am wondering if there is a way to limit the amount of ram that replication can use during the replication, this should increase the replication time but hopefully prevent the virtual memory error.

  
  

• Griffonbait

  Can you tell us about your scenario:

  Are the agents running in continuous mode

  How many change are being replicated for a given sync

  About how much data (kb, mb, etc.) are being replicated for a given sync Or, how large are the rows in the LOB columns, and how many of these rows are being replicated

  
  

• Hauke

  Yunjing,

  Thanks for the reply. Is there a way to currently reduce the amount of data being transferred per sync. For example I know that there is 100 rows with 500kb on each row and I now know that 20 rows will work. Is there a way to have sql start replication do only 20 then stop and start again I know this is a long shot but needed to ask because we're close to the end of the production schedule and my boss is going to freak already. Thanks.

  John

  
  

• skyrancher

  Here refers to what our dev said:

  "Although it is expected that the more data you attempt to sync in one batch the more memory you will need, in the scenario below we are trying to send 100 rows of 500K each and are using 900MB of memory. It seems like there is a bug in the amount of memory we are allocating. We shouldn’t need 20 times the size of data. I will like to debug this further. What we should document is the multiplication factor that customer can use against their data to get a feel of how much memory they will need on their system"

  Our dev is working on this memory issue. Meanwhile, we are also going to document it.

  The workaround so far is to somehow send less amount of data per sync. For example, 100 rows with each row 500KB will cause running out of memory, but 20 rows with each row 500KB to update 5 times will work well.

  Hope these infor will be helpful for you.

  Thanks

  Yunjing

  
  
  

• scottj1234

  Greg,

  Yes we have lob in the subscriber database. We have also setup filter for the lob table to where it only replicates the host_name() information. @stream_blob_columns is set to 1 for the article that contains the lobs.

  John

  
  

• Carlos Iriarte

  Is this the error thrown by the merge agent At the subscriber Do you suspect a memory leak anywhere
  

• Axoren

  How long are these syncs, minutes, hours Are the agents running in continuos mode

  If not, are you replicating LOB type columns, like text, ntext, image, etc. If so, do you have @stream_blob_columns enabled in sp_addmergearticle

  
  

• hendrik swanepoel

  Greg,

  We fire off replication either from command line or a .net application. We're just using the cmd line for testing purposes. We are blobbing in images that range from 200k up to 1 mb. Each subscriber could end up entering between 20 and 200 photos a day. So each replication session could have up to 200 mb of blob in a session. Is this possible If not what is the maximum per session, or is there a way to break it up into multiple sessions

  
  

• Big TC

  Ok, then you should trying turning off @stream_blob_columns. We don't recommend enabling this parameter unless the row size for the blob columns are in MBs. If it's small enough, then try disabling this option.

  
  

• David Palmer

  Greg,

  The ram monger is replmerge.exe. The cpu usage is sporadic as normal. Jumps between 10% to 95% at times. Nothing stays are 100% bogging down the cd. The peak memory maxed at 699,692k vm size 1,281,524k threads ranged from 5 to 11. So it looks like the replication is just sucking all of the ram away. Is there a way to limit the ram usage Doing this might solve the problem but increase the replication time. Your thoughts

  John

  Thanks for the help so far

  
  
• Keep getting Virtual Memory errors