Durable and non-Durable Workflows in same Workflow Runtime

Hi,

I've got a question about supporting workflow instances that are both durable and non-durable within the same runtime. As I understand it if a workflow persistence service exists in the service container then the engine will call it for every workflow instance. If it doesn't exist, then obviously workflow instances are not persisted.

If I want to have a mixed model where I could somehow 'mark' a particular workflow as durable, in other words, use the persistence service whereas another workflow would be run in-memory without persistence, how would I go about it

I thought about a combined database/in-memory persistence service, however, it would need to determine whether the workflow type was a durable workflow. It could only do that by some property or attribute, but this would only exist on a derived root workflow type that was under my control.

Or, I could create two workflow runtime's, one with a persistence service and one without and in the host make the decision as to which runtime (persistence service or no-persistence service) receives the message based on the message's durability property.

If I wish to create both a durable, high latency and non-durable, low latency environment for workflows, it appears I don't quite the flexibility for that at the moment in WF.

Can anyone tell me whether I'm wrong, and if so, point me in the right direction as to an approach that solves this problem

Many thanks,

Paul



Answer this question

Durable and non-Durable Workflows in same Workflow Runtime

  • anish77

    Thanks for the replies.

    If I create a combined in-memory/database persistence service I have to have workflows being unloaded when idle (for efficient resource usage), therefore if I persist to memory in a load balanced environment, then the in-memory activity representation has to sit in a shared state service to be accessible from multiple runtimes.

    If I set UnloadOnIdle to False in the persistence service I still need to detect when a workflow has idled (will the WorkflowIdled event on the runtime give me this even when UnloadOnIdle has been set to false ) and then manually Unload the workflow if it is a durable workflow or ignore if it is a non-durable workflow.

    Alternatively, I just start two runtimes - one with a persistence service and one without. It is a much simpler solution.

    Cheers

    Paul


  • Helen Cool Granny

    Er, another question. Can you block a workflow from unloading If, instead of writing to memory, it actually just told the engine that this workflow shouldn't be unloaded, that would be better than potentially bloating memory...
  • MichaelBrietzke

    When you start your workflows, you could store the InstanceIds that you want to be durable in some collection, and then handle the WorkflowIdled event in the host. Check the InstanceId of the idled workflow with the saved collection of Durable Workflow Guids, and explicitly persist the desired Workflows with a call to TryUnload.

    Would a basic solution meet you rneeds If not please provide additional details of your scenario and we might be able to have some additional suggestions.

    Thanks,

    Steve Danielson [Microsoft]
    This posting is provided "AS IS" with no warranties, and confers no rights.


  • Suryanarayana

    Hi Sonali,

    Thx mate for the replies.

    Yes, when I meant ignore - I actually meant in my custom persistence service I will check to see if it's a non-durable workflow and if it is, just return.

    Cheers

    Paul


  • Kabouter

    Paul,

    Unloading removes the workflow from the memory and stores it in the persistenceservice if available. If no persistence is added your workflow will always remain in memory. If you have added persistence service and if you don't want to unload(i.e. not remove workflow from memory), you can set the flag unloadonidle to false in the constructor of the persistence service.

    For you scenario, I think you should take the custom persistence service approach you have mentioned in your post. There is a sample for custom persistence service in WF docs for your reference.

    Another option to support your scenario using sqlworkflowpersistence is to
    1. Mark unloadonildle = false as suggested by steve. This will take care of the persistence when workflow idles due to delay or events.
    2. Make sure, your workflow does not have a TransactionScope, or a customactivity with PersistOnClose attribute. Because if these activities are present in the workflow and persistence service is added, workflow will be persisted when they complete.

    Hope this helps,
    Sonali



  • JonasBergstrom

    Thanks for the reply Steve,

    It's certainly an approach. I guess I'm thinking along the lines of something a little more comprehensive (even better, natively supported ;-)).

    I could create a persistence service that works in a dual mode. On the one hand if the workflow is durable it could write to a database, and on the other hand if the workflow is non-durable, it could write to memory. I could provide a DurableService that does as you suggest and stores the workflow instance id's that must be durable and then query the service from the persistence service and either write to the database or memory depending on whether the DurableService knows of the workflow instance.

    That's certainly given me some ideas. I was thinking about the dual WF runtime route, but that approach seems better from an architectural viewpoint.

    What do you think

    Cheers

    Paul


  • LadyASPX

    One more option is to create a custom persistence service by deriving from sqlworkflowpersistenceservice and override Load and Save methods. In these methods you can decide which workflow type should be persisted and which should be ignored by using some book keeping mechanism. If you want to persist, simply call methods on base class otherwise return.

    WF docs include a sample of custom persistence service



  • dgf

    Thanks, much appreciated - totally clear now :)

    If I have UnloadOnIdle set to False, then even though there may be a persist operation for a transaction scope activity or a custom activity with PersistOnClose attribute, I can still use the same approach to the WorkflowIdled event and check to see if the workflow is non-durable and ignore the persistence request.

    Cheers

    Paul


  • Tim Parker

    Paul,

    You will not be able to ignore persistence request if workflow contains transactionscope etc. Runtime does not give you any control before it persists the workflow due to completion of TransactionScope or PersistOnClose custom acitivities. WorkflowIdled handler will be called only if workflow goes in idle state and not before persisting the workflow.



  • chris seddon

    Hi Paul,

    The runtime will raise the WorkflowIdled event regardless of the settings of any services that you may have loaded such as a Persistence service. When you use this approach, then your second paragraph above is right on:

    "If I set UnloadOnIdle to False in the persistence service I still need to detect when a workflow has idled (will the WorkflowIdled event on the runtime give me this even when UnloadOnIdle has been set to false ) and then manually Unload the workflow if it is a durable workflow or ignore if it is a non-durable workflow."

    The code would look like this:

    workflowRuntime.WorkflowIdled += delegate(object sender, WorkflowEventArgs e)

    {

    // Check the InstanceId, and if it is one you want to persist then:

    e.WorkflowInstance.TryUnload();

    };

    Also keep in mind Sonali's point: from the previous post: "2. Make sure, your workflow does not have a TransactionScope, or a customactivity with PersistOnClose attribute. Because if these activities are present in the workflow and persistence service is added, workflow will be persisted when they complete." If your workflow has any of this then the workflow will be persisted when the transaction is executed or the custom activity with the PersistOnClose attribute completes its execution.

    Thanks,

    Steve Danielson [Microsoft]
    This posting is provided "AS IS" with no warranties, and confers no rights.
    Use of included script samples are subject to the terms specified at http://www.microsoft.com/info/cpyright.htm


  • Durable and non-Durable Workflows in same Workflow Runtime