A very frequent requirement that customers present for their CRM system functionality is that they will get automatic reminders from it when an item recorded into the database requires their attention after a specific date is reached. This might be a hard, physical date like a contract expiring or a softer variable such as a follow-up date on an open opportunity. Since receiving email notifications from various online services is such a ubiquitous pattern for process reminders these days, it’s only natural that the CRM system is expected to behave the same way.
What sounds like a straightforward thing to implement with the help of the Dynamics CRM workflow processes isn’t necessarily such an easy task – at least if you want the reminder workflow to operate correctly under all the possible circumstances. In a scenario where the reminder is set by the user via a value entered into a date field there’s a number of other variables that you should take into consideration besides that single data entry. For a developer who is used to dealing with user inputs and exceptions this may be just another routine task of figuring out the logic surrounding the reminder date variable, but for a functional consultant who typically works more directly with the end users and mainly just points & clicks in the CRM customization menus for building out the required functionality this may not be such familiar territory. I’ll be the first to admit that I struggle with workflow logic a lot.
Given how much time I’ve spent wondering “how should I do this” when it comes to CRM workflows, the wording of this blog post’s title is pretty bold. Is there really a single right way to build reminder workflows and do I know nearly enough about them to claim the privilege of defining the proper design pattern for them? This is actually just more of a provocation through which I’m hoping to gather feedback on the inevitable shortcomings of this post via the comments section. Another reason behind this choice of the title is that I stand on the shoulders of giants when it comes to my knowledge about CRM workflows, meaning everything I know about them I’ve read from blogs elsewhere and I’ll try my best to highlight the articles deserving most attention via linking to them as we move forward.
In this post I’ll use a scenario where the opportunity record has been extended with a custom follow-up date field. Entering a date into the field is done by any user who has access to update the opportunity. When the follow-up date is reached, a reminder should be sent to the owner of the opportunity, but only if the record is still in an open status, since won or lost opportunities don’t require reminders anymore. Sounds logical? Good, then let’s start working on implementing all the required business logic onto the Dynamics CRM workflow editor canvas.
Start conditions
The first thing we need to decide on when designing workflows is defining the criteria on when should we be performing any actions to begin with. In our scenario, any moment a user could enter a value into our reminder date field is a potential trigger for our workflow logic, so we should have the process started both on the create event of the record as well as the change event of the field in question.
Since we’re dealing with an optional field that doesn’t necessarily contain any value, we should check this right away. It’s not enough that there is a value in the field, but it also needs to be something we can act upon. There are two general things that interest us: 1) is the record sill active (meaning does it require any reminders) and 2) is the reminder date in the future. There’s no point in us sending an email to a user right away if he or she mistakenly enters a date that’s passed last month already, so better to just ignore any such input values and do nothing in this case.
Wait conditions and stopping the workflow
If we’ve determined that there is a valid reason for our workflow to take note of the event to which it has been attached to, the next step is to… just wait. No, I didn’t mean you can just start slacking away with your workflow design! Waiting in itself is a topic deserving your full attention as a CRM customizer, since there are more than just one type of waits in the platform we’re working with. Read this great article by Paul Nieuwelaar for an in-depth discussion on the different options available.
There are multiple possible outcomes that should cause our workflow process to wake up. It’s not only the events when a follow-up action should be performed but also any path that leads to a state where the workflow is no longer required. After all, you really don’t want to have an ever increasing number of workflow instances left forever waiting in your CRM system, hogging up resources unnecessarily. So, in our scenario, whenever the status of an opportunity changes to something else than “open” before the reminder date is reached, we’ll want to stop our workflow.
This brings up another best practice regarding Dynamics CRM workflows. We have the option of stopping a workflow with the status values of “Succeeded” or “Cancelled”. As a general rule of thumb, you should always choose to stop your workflow as “Succeeded” unless you have a good reason to do otherwise. This is because a “Cancelled” value is considered an exception in the workflow process job execution and it will leave a permanent record into the system for debugging purposes. If the stopping is just business as usual for your workflow logic, let the system job be deleted by choosing the “Succeeded” option. Only use the “Cancelled” value if you intend to go through and review all the cancelled process instances when testing or debugging your workflows.
Change of date
Since our trigger field is just a normal date field on the opportunity form, we need to take into consideration the fact that a user may at any stage go and update the date value. For example, when the opportunity in question gets delayed due to lack of actions from the customer’s decision makers, the opportunity follow-up reminder would naturally also get postponed to a later date. Whether a reminder has already been sent for the record or not doesn’t actually matter to us, since what we’re dealing with in this scenario is a reusable reminder field that should always work reliably and deliver a notification at the time specified by the user.
Since our start conditions were tied to not just the creation of a new record but also any changes made to the follow-up date field, this will create a new instance of the workflow process whenever the field value is updated. We don’t want to have multiple concurrent waiting workflows for the same record and we most certainly don’t want to send more than one reminder at the follow-up date, so we need a logic to get rid of the old process instance when a new one is created. But how do we tell the existing workflow that a new version has been created?
There’s one additional “gotcha” in this situation that relates to how Dynamics CRM deals with null values in workflow parameters. Suppose the user determines that no follow-up reminders are necessary for the record anymore and clears the follow-up date field on the form. The workflow process that tries to evaluate this value will throw an exception, as illustrated in this post by Allan Varcoe. Not only would this cause issues with having cancelled workflow instances piling up in the system, it would also stop us from executing any other logic in such a workflow.
Luckily there’s a workaround to this issue and it fits our needs perfectly. As described by Allan, what we can do is store the current value of the follow-up date into a hidden field when we start our workflow. We then use the cached value in this field to compare it with any subsequent new values entered by the user into the actual follow-up date field. So, on the very first line of our workflow process we would add the following update action:
Then at the waiting stage, in addition to waiting for the opportunity status change event, we’ll add a parallel wait branch that will monitor the changes in the follow-up date field. If at some point in time the field value on the opportunity record no longer matches the one that was cached into our hidden field at the start of this workflow instance, we know that the current workflow instance is no longer required and we can gracefully terminate it.
This will make sure that there is always a maximum of one workflow instance waiting for the specific follow-up date on a single record. We could have built a similar type of logic also in the form of a workflow instance counter that would have always incremented a field value upon the start of the workflow, but due to the possible exceptions being thrown if a user clears the date field, this is a much better approach for us to take.
Sending out the reminder
If an opportunity record passes through all of these conditional gates that we’ve set up, it most likely will be a valid target for a reminder message. Alternatively, we may have simply missed some possible outcome and need to add further conditions to the workflow rule. This is one reason why it often makes sense to build the business logic in a format of continuous elimination, meaning you keep adding criteria that filters out any scenarios where you don’t want the action to be performed (in which case you just stop the workflow as succeeded with no actions taken) and wait until the very end to actually perform the work that you originally set out to do with the workflow.
In this example we’ll send out an email message to the opportunity owner when the reminder date is reached. Now, a question that some of you may have on your mind is “why not just create a related task with a deadline as the reminder?” While it’s true that a very simple way for dealing with the reminder requirement would have been to add a new task record when a date value is entered on the opportunity record, this would make it very complex for us to later on validate the need for the reminder if any of the outcomes we’ve discussed above would have taken place (change of status, new dates being entered etc.). This is why in general I’m not in favor of creating any task records in advance to serve as a reminder, but rather prefer to wait until the last minute to see if an action is really required. If you wish, you can of course add a task record alongside the email notification to act as a more persistent reminder.
While you might think that the Send Email action would be a very straightforward thing to implement in a workflow, I recommend you to take a different route and instead add a separate child workflow that will take care of the actual email sending. Why? Because this will provide an additional “shield” for any events that may occur between the time the workflow instance is created and the moment the actual email should be sent out. For example, if you wish to change any properties in the email message (changes in the organization’s processes, new sales director wants to be included as CC in all the reminders etc.), then any waiting workflow instances may have already cached a copy of the original definition, making it difficult to change the process on the fly. On-demand child workflows will not suffer from such handicap.
By having the email send step included in a child workflow that is executed on demand rather than after a long waiting period, we are effectively implementing the dispatcher workflow pattern described by Dave Berry. To understand all the benefits of this pattern, I recommend you to review Dave’s article, as well as this extremely thorough piece by Adam Vero on when and how to use child workflows in Dynamics CRM. These things are far from obvious to the casual CRM customizer, but after reading the real life examples presented by Dave and Adam you’ll most likely want to start using child workflows in many more scenarios than what you’ve previously thought.
Putting it all together
We’ve now managed to build a workflow process for sending out the opportunity follow-up reminder, which will look something like this:
The child workflow will consist of just the email sending step, so that’s not covered in the screenshot above. Since interpreting the logic behind a workflow from the steps configured in the workflow editor can sometimes be a bit challenging, I’ve also drawn the logical flow of the process into a series of steps illustrated in the following picture:
As always, there’s certainly more than one “right way” of designing a workflow process to implement a custom business logic in Microsoft Dynamics CRM. Hopefully this example has given you some ideas of how to make your reminder workflows cover the possible outcomes in a simple date field based scenario, allowing you to further extend the model to meet the requirements of specific business processes.
Jukka, a great and thorough article as always, and thanks for the link to my child workflows article, as well as those from Dave and Allan that I had not previously read.
You prompted for comments, so just a couple of thoughts on the very slight changes I would make (your overall approach and logic is sound):
– Rather than copy the follow-up date to the hidden field as the first step, I would do this as the first step after validation. If I am not going to do anything in the workflow because it fails validation, I do not want to update the record in any way at all, as that impacts the modified by / on fields. For troubleshooting and analysis it might be useful to be able to show which user was the last to change the record, for example to show that the reminder date selected was already in the past.
– This is also because as a general principle I always have a validation condition as the first step of any workflow, so i instinctively look to the first line for a condition, and in this case I don’t see one.
– Another “best practice” I tend to adopt is to always do my validation backwards – IF the current conditions are NOT appropriate, then STOP. As you mention, this gives me the ability to easily choose to stop as Cancelled during troubleshooting so that these stand out (I agree with your general principle that Succeeded is almost always the better choice long term).
– When doing this “reverse” validation check, don’t use “Otherwise” to do the remaining steps, just add as the next step under the IF clause. The workflow will just “run on” to this step if the condition is not met, and the workflow does not stop. By doing this, you make it much easier to change things later, and importantly avoid using up a level of nesting (indentation) before you have even started. In really complex workflows with multiple conditions and sub-conditions, you can only nest to five levels, so avoiding using one from the start is a good practice.
– The one weak spot in your logic is if the user closes an Opportunity and then re-opens it, you would not have a notification workflow running. I would probably take out the parallel wait that shuts down the workflow on change of status, and instead add a condition after it reaches the follow-up date to see that it is still open before doing the other steps to send the email. This does mean your workflows might be left in a waiting state even after the Opp is closed, only to later wake up and then do nothing at all, but this is not really a big deal. The only alternative would be to trigger a new workflow on change of status, check if was changed to being Open, then perhaps call your main one as a child, but this would mean lots of unneeded workflows being triggered on every normal close of an Opportunity for the very rare occurrence of a re-opening.
– Something I might add to the logic is if the user puts in a follow-up by date in the past, take it out (clear the field). This might help if you want to look for Opportunities that have gone past their follow-up by date (presumably if they are followed up and still open, the date is reset to push it further out to the future). It might also be useful to consider imposing a maximum on the field – if it is set more than a year / month / week in the future (depending on the kind of sales cycles involved), then ignore the user choice and set it to 1 year / month / week after the modified by date (then immediately stop, because another instance of the workflow will just have been triggered when we changed the date).
As you indicated at the beginning of your article, you might not want or need to use an email alert for this scenario anyway – but that would not have made for a very interesting discussion about how to do it!
In this case I would probably try to steer a customer to simply using a view on a dashboard showing open Opportunities with a follow-up date, sorted with the first at the top. I discussed how I try to make the judgement between using notification emails and other methods such as dashboards in my article http://blog.crmguru.co.uk/2011/10/10/dont-let-crm-overload-users-email-inboxes
Excellent comments, Adam! I kind of knew I could count on you to expand the article further 😉
I fully agree with the order of the first steps. The idea for the example reminder workflow actually started from the discovery made by Allan about the workaround for the null values, so this was the first step that I added into the process. Then due to the default settings of the “Add Step” action, the validation step ended up as the second line. Now, as we know, one of the big caveats with the workflow editor is that you can’t change the order of existing steps, so perhaps this serves as a good reminder. Typically I always start the workflows with a dummy validation step (so, even in the child workflow for email send I’ve added a “if opportunity is open” condition) to allow for later editing of the rule.
Considering the reopening scenario is another good point. It sort of ties into the discussion on whether to clear the field when invalid values are entered, as I think calling for a new value to be entered in this case might be in order. It’s a shame that the current CRM 2013 Business Rules don’t yet allow us to directly perform many of the required date validation actions on the form level (by referencing today’s date rather than a field value), as the optimal way really would be to show an error message on the field level in real time once the user clicks or tabs away from the field. Server-side validation via synchronous workflows is of course a decent workaround for configuring these type of validations without custom code.
As for the right tool to deliver the information to a user about a CRM record requiring actions from them, email notifications always require a fine balance and can turn against themselves quite easily. Especially if the logic of the workflow behind the notifications has not been properly thought out and a user receives a large quantity of notifications under incorrect circumstances, this can really diminish the perceived value of the system. At the same time, email has become a channel that even all the new generation social tools and networks primarily rely on to draw the user back into the application.
One method to ensure that the users don’t ignore important events in CRM but still don’t drift into a mode of relying 100% on being emailed when they need to log into CRM could be to formulate the notification message in a different way. Instead of saying “this record needs to be updated” and then providing a direct URL to that record, perhaps the link included in the email should rather take them into a dashboard that presents the user with all of the items in the system that are either past their due date or will require actions within the next week. It’s still easy enough to get to the record with 2 clicks (assuming the dashboard views wouldn’t be filled with tens of neglected records) but at the same time you get a chance to educate them about the additional value that accessing CRM directly can provide them, by showing charts and other related information that would potentially be of interest to them.
One of the reasons I tend to use the “reverse condition and stop” pattern, is that you can use it even if you forgot to add a condition into a workflow (or inherit workflows from someone else). You can insert the step before the first step of the workflow, whereas using a traditional condition is, as you indicate, harder to retro-fit to a workflow because you can’t re-order the steps or ‘indent’ them into a new condition.
I like your idea of providing a link to a dashboard or a view where the use can see other similar items. I often build “non compliance” dashboards that show these kind of things all in one place. They should usually be empty, so a user can see at a glance if there is something they need to address. These can range from simple things such as Opportunities past their close date, or one stage in a process such as “records awaiting approval”.
Jukka, top article as usual.
Adam the ‘reverse condition and stop’ tip is spot on.
I’m going to approach this business problem from a completely different angle. I think the best approach is to have a scheduler look for these events & then execute a workflow to perform the desired action.
So we have the following pattern,
-Scheduler to execute on a fixed time. e.g. hourly, daily, weekly, etc.
-It runs a fetch-xml command to find records that meet your desired conditions (i.e. followup date is today)
-For each record found execute a workflow to perform the desired action (i.e. send email)
Some of the advantages are that there is no need for complicated workflow logic to handle all the edge cases like null values on the followup date. Performance, when you have medium to large CRM systems with thousands to millions of records this method is much more scale-able & its performance is in orders of magnitude faster that a single workflow for each record that you need to check.
Lucas Alexander & Yaniv Arditi have written 2 good articles on this approach,
http://www.alexanderdevelopment.net/post/2013/05/19/Scheduling-recurring-Dynamics-CRM-workflows-with-FetchXML
http://blogs.microsoft.co.il/rdt/2012/09/28/asynchronous-batch-process-pattern-part-1/
Hope that makes sense.
Shamless Plug
North52 has a scheduler component as part of BPA, which you can use as part of the free basic edition to implement this design pattern
http://www.north52.com/business-process-activities/scheduler/
HI Jukka, excellent article. Have you considered updating it (particularly the workflow screen shot and your illustrated flow) to account for any improvements you would take on board from Adam? That would be greatly appreciated.
Thanks for your comment, Steve! I think that it would be somewhat difficult to change the original post while preserving the context of the follow-up comments and discussion that has been added to it. Personally I see the social nature of blog format content as its most powerful aspect when compared to traditional “static” content pages or books. In a way, I just write the first comment and the readers’ contribution makes the story complete.
Revisiting the topic of reminder/notification workflow design practices in a whole new blog post is certainly something that I might do in the future. For example, the implementation methods presented by John (batch processing of records based on a schedule) are another area that probably isn’t intuitive enough for many people who are requested to configure such processes for their Dynamics CRM organizations.
Hello Jukka,
I have always found your insights to be excellent, thank you for the time you take in sharing them with the rest of the world!
This article is great and was very similar to what I am working on, so thanks for the validation that it’s the right direction. The only difference I’m trying is to base it off of the Estimated End Date.
In your example above, do you have a second workflow running against the Estimated or Actual End Date fields to cancel the opportunity when either of those dates is reached?
Thanks, Rob! Regarding your question, I’m not quite sure I understand the business requirement behind your workflow. Are you trying to automatically close the opportunity if the estimated close date is reached before the opportunity is closed or updated by the user? Or are you looking to send a reminder to the user if the estimated close date is reached and the opportunity remains open?
Any which way, there shouldn’t be a need for having more than one running workflow if there is just one target outcome for it. Whenever the criteria is not met, you should have a parallel wait than will stop the workflow as succeeded when that particular event occurs.
Sorry for the confusion. There are two date triggers that I’m working on, one for follow-up reminders and the other is a close action. I find that if I don’t close opportunities after x number of days, they grow stale do not accurately reflect the sales pipeline. So when an opportunity is created, I set an Estimated Closed Date on the opportunity using some business rules around how long it should be open.
The business wants two reminders, one at 14 days before Est Closed Date and one at 7 days before. If nothing happens, close the opportunity on the Estimated Close Date with a certain status. This can feed into the sales rep’s metrics.
This is what I wound up doing, it may be too complicated but I think it will work (testing it now). I have three main workflows: one base and two child. First the base:
-Triggers on Status change or Field change on Estimated Closed Date
1. Check conditions to see if opportunity is open, if not cancel the workflow.
2. Copy the estimated end date to a hidden date
3. Check if the execution-time is >= 7 days before Estimated Closed Date. If so, start the -7 day monitor workflow that only monitors for a change in status, change in the date, or waits until the date and then closes it.
4. Check if the execution time is >=14 days before Estimated Closed Date. If so, start the -14 day monitor workflow that only monitors for a change in status, change in the date, or waits until 7 days before the date. Fire a child process to send 7 day follow up, then fire the 7 day monitor.
If all checks pass, the execution date is > 14 days, so go into thee wait conditions:
1. Change in status – if so, cancel this. A new instance of the workflow will start with the change.
2. Change in Estimated Closed Date – if so, cancel this. A new instance of the workflow will start with the change.
3. Execution date is 14 days before Estimated Closed Date. When this happens:
a. Start 14 day reminder child workflow
b. Start -14 day monitor child workflow
The -14 day monitor is a child process with no triggers. It has 3 wait conditions:
1. Change in status – if so, cancel this. A new instance of the base workflow will start with the change.
2. Change in Estimated Closed Date – if so, cancel this. A new instance of the base workflow will start with the change.
3. Execution date is 7 days before Estimated Closed Date. When this happens:
a. Start 7 day reminder child workflow
b. Start -7 day monitor child workflow
The -7 day monitor is a child process with no triggers. It has 3 wait conditions:
1. Change in status – if so, cancel this. A new instance of the base workflow will start with the change.
2. Change in Estimated Closed Date – if so, cancel this. A new instance of the base workflow will start with the change.
3. Execution date = Estimated Closed Date. When this happens, close the opportunity.
So I am sitting here attempting to figure out what the areas that are not shown or even explained at in order to end start this workflow. I do apologize if this question was somehow answered within one of the comments above, I just do not have the time to read through all of them….it’s 4:18 on a Friday! So this is any place you can see “Select” or “Set Properties”…What are the Properties to set here?
How do I set Task flags in CRM to go to one main administrator if the tasks not completed on time
Gigi, if you want the task ownership to change at a deadline date, you could apply the same workflow pattern shown in this article but use an Assign step instead of sending the email reminder. If you’d rather preserve the original task owner information, then how about creating a task view to the CRM administrator for viewing all open tasks where the due date is in the past?
Task view sounds ideal however if sales staff do not contact their leads within the prescribed time how will the sales manager see an overview of this not done on a weekly basis – so i need a task and lead report showing inactivity
Thank you Jukka do you think that this is possible
Hi Jukka,
I followed the steps and created a workflow with wait and parallel timeout. For testing purpose I kept Timeout condition as 1 min after the Case created on. Expecting that it will do some operation after 1 min, nothing happened. Am I missing something over here.
Thanks
Hi is there a a way to use todays date or getdate in the condition.
I want my condition to be where todays date is less than or equal to follow up by date – 2 days
Thanks
John
John, if you need the time of when the workflow process criteria is first evaluated, use the Process Execution Time in your condition.