How does one properly throttle a PeerNet network when messages are pouring in faster than the receiving nodes can process them
I'm trying to prove whether or not PeerChannel would make a good foundation for publish-subscribe style messaging that is reliable and robust enough for an enterprise.
My initial testing has been very positive. PeerChannel is looking to be super-reliable so far.
Sending 10,000 messages in a 5-node configuration:
0 message loss, 2,500 msgs/min to 4,000 msgs/min per receiving node
Sending 100,000 messages in a 5-node configuration:
0 message loss, 2,500 msgs/min to 4,000 msgs/min per receiving node
Send 1,000,000 messages in a 20-node configuration
Fine for first 2 hours (about 175,000 messages) then all systems run out of memory
Given the last result above, it seems like I need to hold back message sending when the network is overloaded. But how do I detect that condition What is available for a sender or receiver to determine how big the backlog of messages is
PeerChannel throttling controls?
Annie24372
David, this is great data! Thank you for sending!
Can you tell us:
1. What is the size of messages you are sending
2. Is there one sender or multiple senders
3. Does the sender also go out of memory in addtion to the receivers in the 1M test
4. Is there any data cache you are building up in the application as well
5. What is the machine configuration of receivers/senders
6. How many nodes are you running per machine
Thanks,
Ravi
Donna M-T
Rich Hanbidge
I apologize that this question dropped off our radar. As your question refers to a build that we are no longer tracking and we have made significant changes to the product since then, could you please verify whether your issue still exists with the most recent July CTP of the .NET Framework 3.0. If it does and you still need assistance with it, please reply here.
Thank you,
Clemens
pallavi
Hi Madhu,
what is the message size you are using and what is the text length that you are setting it to
you may be hitting a reader limitation on the strings in the incoming messages: try setting NetPeerTcpBinding.ReaderQuotas.MaxStringContentLength appropriately.
Hope this helps,
-Ram Pamulapati
DaSteph
gpspilot
Hi Andy. I'll continue to post results as we progress (particularly when we have problems that need solving!). As for your web site question, Neudesic (a Microsoft Gold partner) does blog a lot of useful information about WinFX and other MS technologies (blogs.neudesic.com) and we do expect to launch a WinFX site ourselves in the near future where we'll post this kind of data as well as articles and code samples.
David
Sergey Barskiy
claudia galimberti
Yes - you are correct. This is not well documented, but using svcconfigeditor.exe (which is in the ~\Microsoft Visual Studio 8\Common7\IDE - for anyone with the same problem) folder and increasing the ReaderQuota on the binding fixed the problem - thanks.
lololon
Hi Ravi. I've posted my test data below in more complete form and it should answer your questions. I'm just getting started and plan to provide a lot more testing with varying messages sizes and increasing numbers of machines to prove enterprise worthiness of this approach... but first I need to solve this out of memory problem. Any thoughts on how I can manage throughput of the network Our use of PeerChannel hinges on solving that problem.
Thanks,
David