Tuesday, May 24, 2011

Scheduling recurring workflows in CRM

Quite often we have a business process that needs to execute recurrently every month, week, etc; but there is no Out-of-the-Box scheduling engine in CRM, so that leaves us with a few options:
  1. Host a service that implements the scheduling functionality and then have this service call into the CRM web service to perform actions regularly.
  2. Leverage the CRM workflow “wait” step.
  3. Alter the recurrence patterns of CRM asynchronous operations (unsupported).
I will explain the second option in this post. As an example scenario I have the following requirements: “The first day of the month and email should be received with a summary of all the accounts created last month”.

I will start with the assumption that I have already created a custom workflow activity that retrieves the accounts created last month and outputs the text necessary for my email. Now I need to follow these steps to make the workflow recurring:


1.  Define triggers:

Define the workflow as On-Demand and Child workflow (not automatic triggers)
  


2.  Define process:
 
Define the body of the workflow to represent the set of actions that must occur every month:



3.  Add the Wait:

Now you need a way to tell the workflow engine to repeat the process every month. To do so, I will first insert a wait step that will tell the workflow engine to wait for 1 month:


After the step, I need a way to “restart” the workflow. Since there are no while/for loops in the workflow designer, we must find another solution. I will simply create a recurrence pattern by having an “Execute child workflow” step which executes itself. Because the workflow is defined as a child workflow, it will call itself every month. After I activate the workflow it looks like this:



4.  Trigger the workflow once
 
The first time that the workflow executes, it must be triggered manually, after that it will schedule itself monthly from the time that it was executed manually. Since it is defined as an on-demand workflow, you can simply navigate to your primary entity and trigger the workflow manually.



5.  Verify

Once your workflow has been triggered for the first time, it should appear in the system jobs grid, waiting for the next month to resume execution:
Note that “month” is not the same as 30 days, it actually uses the calendar month definition, since CRM workflow uses the .NET DateTime class. Therefore, if it was first triggered the 1st of the month, it will always trigger the 1st of the month regardless of the number of days in a month.


WARNINGS:
  • Every time you create a recurring workflow, you should carefully consider what resource consumption implications it will have. Recurring operations tend to utilize a large amount of resources which can cause lagging to the CRM Asynchronous processing service.
  • You must select a primary entity for your workflow which will never be deleted, otherwise the entire recurring workflow will be cancelled. For example, you can set the primary entity of your workflow to be systemuser and trigger it the first time regarding your own system user.
  • There is a loop detection mechanism in CRM which would cancel processes/plugins that create infinite loops as the one above. The maximum depth for a recurrence/loop is 8; however, the depth is reset after 1 hour of inactivity (wait). Therefore, the workflow above should work fine because it has 1 month of inactivity so each time it executes the depth will be reset and it will never be cancelled by the loop detection mechanism.

    30 comments:

    1. Hi Gonzalo,
      Thanks for the article, it's really helpful. However, I do have a question for you. I need to send weekly reports regarding opportunities on an automated basis. If I was to send just the link to the report, I can achieve that using your method. But, how am I supposed to achieve it if the reports need to go out in PDF formats. I need to do this because I need to send only the snap-shot of the report.
      Please let me know if you know how to achieve this, using workflow or something like a SQL Reporting Services Scheduler?

      Thanks,
      Ankur

      ReplyDelete
    2. Hi Ankur,

      I am not sure this is the best method for scheduling reports. I suggest you take a look at this post that might be helpful: http://blogs.msdn.com/b/crm/archive/2009/08/05/creating-report-subscriptions-in-microsoft-dynamics-crm-4.aspx

      ReplyDelete
    3. Hi Gonzalo,

      I just discovered the Wait condition in Workflows and it is great. I am working on CRM 2011.
      My goal is to send a follow-up email after 2 weeks without answer from the customer. Then 2 weeks later still without answer, to close the case.
      I wanted to know if it is possible to stop a Workflow that is waiting or does it have to reach the timeout absolutely?

      Thanks for your help,
      Sébastien

      ReplyDelete
    4. In order to cancel "waiting" workflows you must o it manually by selecting the system job and clicking "cancel". However, your workflow can have a timeout of 2 weeks and then in the next step you can insert a "stop workflow" step to cancel it.

      ReplyDelete
    5. I think what Sébastien is asking is if its possible to programatically stop the workflow if the customer has already answered.

      If that's the case, it is possible indeed by using the "Parallel Wait Branch" step.

      He would also have to setup another workflow that completes a custom "Last Customer Response Date" field so the first workflow can wait Two Weeks and "Parallel Wait" till that Field is modified with a new date.
      The workflow will end whenever any of the two conditions is met.

      ReplyDelete
    6. Hi Ruiz,

      Great blog, very innovative and much needed. I have been reading your blog since yesterday time and time + have posted same question at the forum: http://social.microsoft.com/Forums/en-US/crmdevelopment/thread/1674af80-d48f-4a7a-9ef7-e5f30afa0d35

      I must admit, your blog is very popular as everybody is posting your link when it comes to recursive workflow.

      Back to my question, that i have posted at the forum. I have plugin on one my custom entity that does bunch of calculations, i need to execute that update plugin for all the records at the start of the year.

      The timeout = 1 month thing, kicks off when we run it manually (no problem doing it), any chance we can tell the system to say start of the year.

      Sorry my expertise level on workflow is that of a beginner so would really appreciate if you can point me in right direction.

      Thank you
      Sam

      ReplyDelete
    7. Hi Sam,

      Unless you schedule some kind of service (independent from CRM) which will trigger the workflow at the beginning of the year then you'd have no choice but the execute it manually. There is now way to schedule workflows in CRM to start on a specific date-time, you can only postpone the execution to a specific date time.

      A workaround you can explore is to insert an additional "wait" step at the beginning of the workflow which waits until Jan1st. Then you can trigger the workflow manually and it will go to sleep until Jan 1st.

      ReplyDelete
    8. Hi Ruiz,

      How did you set the dynamic value {1 Month} using CRM workflow designer? Actually I need to trigger the workflow on March 1st in each year. Can we use this technique to do that?

      Thanks,
      Thushara M.

      ReplyDelete
    9. To set it to {1 Month} you need to select "Duration" in the form assistant (insted of Before/After). Process Timeout Equals Duration: (1 Month). You can use this technique, you just need to start it once on March1st manually and then tell it to wait 12 months.

      ReplyDelete
    10. Great Post Gonzalo.
      Does this work if we set the interval to 1hour ?? Will CRM not detect the recurring workflow as infinite loop & cancel it??

      ReplyDelete
    11. If you set the time interval for 1 hour then you should be fine. Anything less than 1 hour will be caught by the infinite loop detection and the workflow will be cancelled. To be sure you can configure it to something like 1 hour + 1 minute. After 1 hour of a system job being "inactive", the depth couter is reset so the infinite loop detection can be bypassed.

      ReplyDelete
    12. Hi Gonz,
      I have gone through your post. A great explanation is given by you, which helps to me as I am beginner in CRM.
      Now my question is that we have a scenario which very much like the your example only one thing is that we have added custom date field in account entity and we need to check that date with current system date to send an email alert to user. We want this for all records in account. So by this workflow process can we check the date condition for all records in account entity for each day and if a conditional match found it will take appropriate action.
      Can you help me out for this scenario?
      Thanks,
      Suhas.

      ReplyDelete
    13. Hi Suhas, I'm not sure recurring worklfow is good for your scenario, you might end up drowning your async service if you have daily recurring workflows for every acount in the system. You can simply make your workflow wait for the date you want and then send the email on that date.

      ReplyDelete
    14. Hello Gonzalo,
      Great Idea, Thannks!
      I have only one question... ehat about the new records you didnt select?
      What about new records create later?
      Do I have to execute manually the WF again?

      ReplyDelete
    15. Hi Gonzalo,
      Great post.
      I have similar scenario where I have to run recursive workflow on 1st of every month for multiple records. Like on 1st of the month convert all Account with status "Closed" (custom status field) into Leads. Is it possible to run the same workflow for multiple records and periodically?

      ReplyDelete
    16. You cannot run a workflow periodically on multiple records out-of-the-box. You could use the "Distribute" workflow activity (from Codeplex) to run a workflow on multiple records.

      ReplyDelete
    17. Hi Gonz - can I run a workflow on a custom View every 24 hours?

      ReplyDelete
    18. Hi Gonzalo,

      Nice article.very helpful...thanx

      ReplyDelete
    19. Hi Gonzalo,

      Thank you for your post. Our organization deals with subscriptions. The subscriptions start on the first day of the month. So if a person pays for a subscription on November 22, 2013 then their subscription will start on December 1, 2013.
      Is there a way to capture that process, where if a person pays in the middle of the month that a workflow will update date field to the 1st day in the next month.

      Thank you,
      Scott

      ReplyDelete
    20. on June 28, 2012 at 8:04 PM Gonz wrote :
      Anything less than 1 hour will be caught by the infinite loop detection and the workflow will be cancelled
      But on Warning section Gonz wrote :
      however, the depth is reset after 10 minutes of inactivity.

      So, which is the correct one?
      I've just tried a 11 min. wait and it falied.
      FYI :
      My goal was to perform rather large update of contacts on Online deployment.
      Since sandbox operations (async, sync or custom workflow) all must be registered as sandboxed, 2 min timeout prevent this massive update.
      The solution was using recursive WF, as suggested above. However, 11 min. were not enough to reset the depth parameter.

      ReplyDelete
    21. Sorry the correct time is 1 hour

      ReplyDelete
    22. This comment has been removed by the author.

      ReplyDelete
    23. Gonz;

      I think my situation is similar to Jaydeep's in that I'm working with multiple contacts. We use CRM 2011 Online, and every hour i need to run a workflow on contacts that have been updated. The contacts are brought up in a Personal View, and i think this part can be accomplished by a "Check Condition" in the workflow before it proceeds to the wait routine. The workflow I'm doing just updates the contacts even further. Would I need to talk to our SysAdmins about the "Distribute" Workflow Activity to accomplish this?

      ReplyDelete
    24. Hi Gonzal,
      Thats a very good explanation.
      But when I implement in my environment, my workflows end after I manually start the workflow for the first time. I have two custom entities: Main entity is Calls and the other enitity is task. Calls entity has 1:N relationship with task entity. First my task workflow needs to run every hour and then after 5 mins calls workflow needs to run.
      I've done same like yours and triggered both manually for the first time and they gets succeded after an hour and stops. What am I doing wrong? In the workflow I see Postpone until with the date as today and the time as the hour it stopped running for the first time. Is this something I need to change, if so how?
      Please let me know ..
      Thanks

      ReplyDelete
    25. Hi Gonzalo,

      I have a workflow to be activated on the 1st and 15th of each month. I think there are two options of implementation.

      The first approach is to create two identical workflows that launch on the 1st and 15th of each month and then launch them again one month later.

      The second approach is just to have one workflow and in it, add a code to check the current date and bypass the business logic if it is not the 1st or 15th of each month.

      Which is the best option? Or you have better solution?

      Tiong

      ReplyDelete
    26. Building off of this, I was able to create a perpetually recalculating process - basically my own custom rollup field. Here's my writeup: http://ms-crm.guru/?p=156

      ReplyDelete
    27. Hi Gonzalo,
      I am very impressed with your logic of Recurring WF.Thanks a lot.
      Here my question is,I need to run a custom activity every 15 mins.
      I need your clarification on this.
      Anything less than 1 hour will be caught by the infinite loop detection and the workflow will be cancelled?
      Please let me know.I am using CRM 2016 online.
      Thanks in Advance.

      ReplyDelete
      Replies
      1. yes, at least 1 hour is the minimum.

        Delete
    28. This comment has been removed by a blog administrator.

      ReplyDelete