Monday, September 14, 2015

CRM 2016 announced! My favorite features to look out for


You might have seen the press release (, CRM 2016 is no longer under NDA and will be released by Q4 of this calendar year! Below is a very quick summary of favorite features I have extracted very high-level (in bold are my biased favorite features!) but if you’d like to read more about it, I suggest you review the release preview guide:




1.    Full offline support to create, update and delete records while offline. *
2.    Open SharePoint/OneDrive documents from CRM mobile app.
3.    App-to-app deep linking
4.    New UI controls and ability to bind data fields to UI controls
5.    Task bases experiences: Show relevant data from multiple records in a single screen for a given task*
6.    Preview phone/tablet experience from browser while designing forms.
7.    Full web resource (IFRAME) support in mobile.
8.    Integration with Intune for secure CRM data with Mobile Application Management (MAM) in BYOD scenarios.


1.    Data encryption for CRM server
2.    New data centers in Canada (Toronto and Quebec City) and India
3.    Bulk Data Loader tool


1.    Enhanced modern API: REST, OAuth 2, JSON, OData v4.
2.    Next Generation Search: Multi-entity search with relevance ranking*
3.    Azure Machine Learning: Easily configure ML integration for product cross sell recommendations*


1.    CRM app for Oultook (PC / Mac / any browser) *
2.    Excel integration including mobile/tablets and Excel templates for viewing sales data
3.    Consolidated document storage integration from SharePoint, OneDrive and O365 groups.
4.    Document Generation: in one click generate Word/Excel documents from CRM data (orders, invoices, etc.)
5.    Cortana integration*
6.    Single- and Multi-stream service hubs dashboards
7.    Re-designed forms for cases, accounts, contacts and activities with reduced clicks, wall filtering, quick actions on related records.
8.    External Party Access: Give access to your CRM data to third party (customers, partners, etc.)
9.    Enhanced knowledge management curation process.
10.    CRM surveys sent to external people (email/web). Results and analysis captured in CRM. (requires Azure to host feedback page)
11.    Field Service Capabilities (FieldOne acquisition)


1.    Enabled SMS channel for marketing campaigns (inbound/outbound) in MDM
2.    HTML email rich editor in MDM


1.    Centralized auditing configuration
2.    OOB integration with Azure 3rd party systems like HDInsights
3.    Stability and Performance enhancements
4.    Simplified install & upgrade experience. Custom DLLs distributed through CRM server configuration.
5.    PowerBI dashboard


1.    Feeds from custom sources (RSS) and non-public sources (Yammer)
2.    “Intelligent Social”: Target accounts and share content, detect potential leads vs. cases
3.    Improved sentiment analysis by domain-specific models.
4.    Groups and custom roles
5.    Social Center: Post to Facebook/Twitter with rich multimedia content from social center.

* Only available in CRM Online.

Monday, August 24, 2015

Agile CRM development with SCRUM

I recently worked in an agile CRM project as the scrum master and was able to learn some good tips of using scrum for CRM development.

I am of the belief that the waterfall days are coming to an end, mostly due to its unsustainable release cadence, inefficiency when reacting to the 'unknown' and the large amount of waste that goes towards tight controls and project management as opposed to focusing effort on product quality and continuous business value generation. In the world of CRM implementations this becomes even more evident because of various factors such as the ability of the platform to quickly produce business value and adapt in an iterative manner, even when the final outcome is yet to be discovered. CRM can be configured very quickly which allows you to achieve valuable work (increment) in iterations (sprints) that can be as short as 1-2 weeks. The fact that you can have stakeholders provide early feedback continuously (sprint reviews) allows you to adapt very quickly to changing demands and gives your users a chance to get familiar with CRM early on. Continuous feedback leads to more valuable user feedback because users have something tangible to review instead of having to imagine what everything will look like. If you are reading this you probably already understand the value of agile and I don't need to convince you further. Now let's look at the typical challenges we find with agile frameworks like scrum and some ideas from my experience that can make your life easier.

1. Your customer needs to fund the project and to do so they need the cost, scope and schedule upfront. This has been the hardest challenge, specially in the consulting world as organizations are used to think that to manage risk means to transfer to someone else (you) the risk of changes in scope, cost and schedule. This is an antiquated mentality when it comes to complex software development because you cannot plan for everything in advance and expect that the requirements you document today is exactly what you are going to need. The result is typically that consulting companies add a large amount of contingency and charge you more for any change in your original requirements (change requests). The end result is that your customer ends up paying more and not getting the optimal solution. What is worse, is that they will not know that the solution is not really doing what they need until it is too late, too much has been wasted and the budget cannot be increased. This is by far the main reason I see waterfall projects fail. But what can you do about it if your customer's culture is waterfall and they want a fixed-everything upfront before they can "fund the project"? I think the magic word here is 'trust'. The best you can do (other than finding another customer that understands that we are not in the 90s) is to earn their trust. Certainly this does not happen overnight but if you are able to build this trust you might eventually be able to propose 'giving it a try' and educating your customer on modern software development practices. The good thing about agile is that you see the results very quickly and you can start by funding a single sprint to see how things go (the bad part is that the first sprint is always the hardest!). One important tip is to make sure your customer trusts you enough to not think that you are just trying to make more money with them, this will not go well and is very far from your objective. In reality your objective is to save them money and increase their probability of success, they need to see that intention clearly. Regarding funding, you can help them switch from the 'fund a project' mentality to the mentality of 'fund a team that is going to give you optimal business value for your project(s)'.

2. You are not sure how to manage the CRM development and test environments for an agile project. This is one where you might have to try a few things and see what works best with your development team. My suggestion is to start with something as simple as possible and empower your development team to decide if they want to make changes to this during sprint retrospectives. The simplest is to have one (yes, one!) shared environment (CRM organization/tenant) for development and test. I have blogged in the past here about how to manage multiple dev and test CRM environments and some good practices about that, but I invalidate my own post when it comes to agile. You want to reduce waste and not have to spend time migrating code, merging solutions and fixing environment synchronization issues. This is why you should at least try to have everyone working on the same environment. You will certainly need some rules such as one developer cannot block the environment for debugging while everybody else needs it; but this is why the daily scrum exists. Make sure you have a daily scrum and that it is actually a 15 min maximum daily scrum and not some status report or defect logging meeting, do not invite anybody outside of your development team to the scrum meeting, this time should be wisely used so developers can synchronize and coordinate activities to be the most efficient. Developers can always have their own sandbox environments but these are not development environments as you should not promote code from sandbox to development.

Regarding test environment I initially had a separate test environment but the development team quickly realized that it was causing too much overhead with very little value. Instead, try to get everyone to test during the sprint (add testing to your definition of done). A very hard thing to do is to bring acceptance testing as part of the sprint and not an ad-hoc activity after the sprint. Get your product owner engaged to do the necessary acceptance testing prior to sprint review, and even consider incorporating UAT as part of your sprint. This is not easy but will give you increased agility and faster release cycles. If you succeed bringing acceptance testing as part of your sprint then it is more clear why you would not need a separate test environment.

3. You don't know how long your sprints should be and how much PBIs you should forecast for a given sprint. I think this is one of the places where CRM has an advantage over custom development, you can do a lot in very little time. Therefore, I would suggest to start with shorter sprints (maybe 2 weeks is a good start). Never do a sprint over 4 weeks or you might end up just doing waterfall! At first you will most likely underestimate and not complete all the PBIs by the end of the first sprint. This is not ideal but is acceptable as long as you don't "extend" the sprint (sprints are time boxed, never extend the end date of a sprint, no matter what). The reality is that at first you might struggle with sprint planning, this is normal, try to forecast how much you can complete in your first sprint by keeping in mind your definition of done (make sure you have one!). So think about how much time it will take you not just to code/configure but also to design, analyze, unit test, document, etc.) each PBI. Additionally you need to work with the product owner to select the most urgent PBIs while keeping a balance for the skills required to deliver the sprint. For example, if you have a dev team of 6 but only 1 person knows SSRS, then you cannot realistically include only SSRS reports in the sprint, even if those reports are the most urgent PBIs. You need to make sure your team skill set is proportionate to the skill set required to deliver the sprint otherwise you might end up with 1-2 very busy individuals while others have a hard time finding how to contribute. Additionally when you forecast your sprint make sure you reserve a 10% of total sprint time to do backlog refinement (most teams fail to do this). There are multiple estimation techniques such as planning poker, however, when forecasting your sprint always ask your team at the end to do a 'gut check' (are we confident to be able to deliver these PBIs within next sprint?). I have found gut checks to be very useful and accurate, everyone must keep in mind the time spent in meetings, backlog refinement, sprint reviews and retrospectives, etc. because all these activities are also part of your sprint, not just the development activities. So in reality you spend only about 80% of your sprint capacity developing PBIs (note that development includes design, code, unit test, etc). One more thing about sprints, try to keep the same duration of sprints and the same start and end day of the week. Eliminate any time between sprints (yes this can be hard). Sprint n+1 should start the day after sprint n finishes, this will give you greater agility, as opposed to having "ventilation" time between sprints to do activities that should actually have been done during the sprint.

4. You are uncertain about how you can incorporate UAT and deployment into the scrum process. This is one of those places where reality sometimes crashes with scrum. If you can get your customer to perform UAT during the sprint and you make that work then you are probably a scrum super ninja that is not learning anything new from this post. So that's the ideal scenario, but we know for various reasons this does not always work. In this case you will need to do as much acceptance testing with the product owner during the sprint as possible and then release to UAT as part of your definition of done. The problem is that your team will probably get interruptions for UAT support as the UAT testers are usually in a completely different cycle as your sprint cycles. You will have to deal with this and account for this during sprint planning, but again the more you can test earlier the better because you will get less UAT defects and distractions. In this case you might need a separate environment for UAT and you would need to forecast in your sprint planning the activities for UAT release. Regarding deployment to production, I have often heard the misunderstanding that in scrum you are supposed to deploy to production at the end of every sprint, that's not correct. You can release to production as often as you want, from multiple times per sprint to once very number of sprints. However, at the end of each sprint, your work should be "potentially releasable" according to the scrum guide, so you shouldn't have broken critical functionality at the end of the sprint, even if you don't plan to release to production. It is a good idea to release as often as you can, try to incorporate the DevOps approach and bring development and operations together as much as possible, this will help you achieve a faster release cycle and a smooth transition to production. Of course this is not something that you will get right from one day to the next, DevOps requires multiple changes and great amount of practice and experience to become efficient (beyond the scope of this post).


Scrum is an empirical process, so at its core it has transparency, inspection and adaptation. Make sure that you pay attention to these 3 values during your various events, the key is not how to successfully setup a scrum project but more about your agility to inspect and adapt your process to produce business value faster and reducing waste, that is your success criteria! There are certainly many other challenges and suggestions, I might blog more about this in the future, but I would conclude with one famous quote from scrum that couldn't be more accurate: scrum is a framework that is easy to understand but very hard to master.

Tuesday, July 21, 2015

How to cascade activate / deactivate (SetState) in CRM

CRM allows for an easy configuration of the cascade behavior for delete, assign, reparent, merge and share. But how can we cascade the status of a record to its children?

This is a problem that I find in many CRM implementations. A typical example is that you want to deactivate all contacts related to an account whenever the account is deactivated. Similarly, if an inactive account is re-activated you might want to re-activate its child records such as the child contacts.

The good news is that I have updated the MSCRMWorkflowUtilities to handle this for you without having to write any code (currently only supported for CRM 2015+).  Simply download and install the solution. Then you can configure any number of cascade behavior for the SetState operation.

You will first need to define a workflow based on the parent entity. In the example above, we would define a workflow on the account entity in order to cascade its status to the child contacts. Then you can insert conditions and as many cascade steps as you want. In my example, I will check if the account is active then activate all the child contacts and if the account is inactive then inactivate all the child contats. However, note that you could insert additional steps if for example you want to deactivate all child contacts and activities then you need to insert one step for each child entity.



For each “Cascade SetState” step you insert, you need to configure 3 parameters:


1. Child Entity Name: The schema name of the child entity to which you wand to cascade the SetState operation. In this case it will be “contact”. You can obtain the schema name from the entity definition:


2. Child Lookup Attribute To Parent: This is the schema name of the attribute (field) of the child entity that links to the parent entity via a lookup. In this case it is “parentcustomerid”. You can obtain this name from the child entity definition by finding the lookup attribute to the parent entity:


3. Target Child State Code: The target state/status to cascade to the child entity. For most cases “1” is for deactivate and “0” is to activate. Note some special entities like case and opportunity are not supported because they don’t use a SetState message. All custom entities are supported.

You can imagine that there can be a performance problem when you try to deactivate a record which has a large number of child records. Therefore, it is recommended that your workflow be asynchronous instead of synchronous. Have fun!

Monday, June 1, 2015

Scalability considerations for CRM / SharePoint integration

SharePoint is probably the most natural and easiest to implement solution for document management for CRM records. However, what happens when you have millions of records in CRM which might have documents?

I recently worked on fairly large document migration solution to be able to associate documents to CRM records (e.g. cases, opportunities). There were a few hundreds of thousands of files amounting to about 50GB of data. So here are a few things to consider


With the out of the box integration you have the choice to structure your folders based on accounts/contacts or simply to create a new library for each entity at the root of the site. It might be more convenient to navigate SharePoint if you can start from an account and then find which are the related cases/opportunities rather than starting with a global list of cases and try to find which cases are related to a specific account. In general, this structure is more convenient, until you consider security segregations.

If you need to segregate SharePoint security in such a way that some users can access documents of a given entity type (e.g. cases) but not be able to access documents of another entity type (e.g. opportunities) then the account/contact hierarchy becomes a problem because you cannot set the security at the library level (all documents would be on the same library) and you would have to do some acrobatics with folder security and inheritance breaking which would be a nightmare to maintain. Instead, if you have each entity type have its own SharePoint library then you can easily grant/deny access to documents of a given entity type.

Nonetheless, remember that with the out of the box CRM/SharePoint integration, there is no security synchronization between CRM and SharePoint. You need to keep this in mind throughout your design. Think for example that a user might not be able to access any opportunities in CRM but the user (if malicious) can always find a way to see the documents associated to CRM opportunities by going to SharePoint (unless you block the user completely from the given SharePoint library).

Also consider that changing the folder hierarchy after would be a major data migration effort so you should really think about what hierarchy makes more sense you’re your situation and consider current or future security requirements.


Imagine that you have 100K cases in your system and you have enabled SharePoint integration for case entity. And perhaps every case has at least one document. In this case, your case library/folder will contain 100K items flat on the same list. This goes well beyond what SharePoint recommendations are for scalability and performance. It is not recommended to go beyond 5000 items (even that is already quite high). Of course you can always implement some sort of archiving or use multiple SharePoint sites depending on some criteria so you split this load. Another [reasonably] easy solution is to further structure your folders by year, quarter or month (or all of these). This way you will not end up with 100K folders under the “cases” folder. Instead, the maximum number of folders you will have under a single folder will be the maximum amount of cases that can be opened in a given month/quarter which might be a more reasonable number.

In the example above, we have 2 new layers: Year and Month. These correspond to the date on which the case was opened. By adding these additional layers, we can now guarantee that there will be no folder with more than a few hundred sub-folders, since we know that we only open a few hundred cases per month.

The downside is that this cannot be done by simple configuration or out of the box integration. This structure would require that you register a plugin on create of case, which will create the document locations and the SharePoint folder for the case being created. Some other disadvantages of this approach is that:

- Every case will now have a folder, even if the case has no documents. This should not be a problem if you know that anyway all cases have documents

- You are creating the SharePoint folder at the same time as you create the case, instead of the OOB behavior which is to create the folder on demand the first time document library is accessed in CRM. This is not necessarily a bad thing though.

- If your plugin is sync and it fails (e.g. SharePoint is down) then it will prevent the creation of the case in CRM. If your plugin is async and it fails, then you need to have a way to recover and create the correct folder when the user tries to access document library for this record.


For sites with large volume of documents, large document sizes or rapidly growing volumes, you might also need to consider how long you have until you start having a performance or limitation problem with things like the max size of your content database, max number of files in the library or simply the maximum size of your site. O365 also has some limitations that you need to review. If you identify that this could be a potential problem you should consider implementing some sort of archival solution which will allow you to keep a link between CRM records and SharePoint files while at the same time optimizing for current and mostly used records. I have heard of people simply changing the CRM site every now and then and creating a new site once the old site is getting too large. I guess any of these strategies would work as long as you have defined a process to scale your site and monitor the volumes regularly. The most important thing is to consider this during your design phase and have identified your approach to handle scalability, even if you might not implement a long term maintenance strategy right away.

Tuesday, April 7, 2015

CRM 2011 Workflow Utilities now available for CRM 2013 and 2015

By popular demand, I have upgraded the CRM 2011 workflow utilities and rebranded it to be compatible with CRM 2013 and CRM 2015.

You might be familiar with the CRM 2011 Workflow Utilities project that I released to Codeplex a few years ago ( By popular demand I have released version 3.0 for CRM 2015 (both for Online and On Premises). You can download the CRM 2015 version here:

Note that if you are still on CRM 2011 or CRM 2013, you must continue to use the release version available here: which has 2 solutions, one for CRM Online and another for CRM OnPremises.

Additionally, I have rebranded the tool from “CRM 2011 Workflow Utilities” to “MSCRM Workflow Utilities” since it supports multiple CRM versions (and hopefully future versions too!).

Workflow utilities is a CRM solution that allows you to extend the CRM workflow design experience by providing the following custom workflow steps available directly from the CRM process designer:

  • Delete record
  • Share or “unshare” a record
  • Insert hyperlink to a CRM record
  • Qualify lead (convert to account/contact/opportunity)
  • Bulk activate / deactivate records (no record count limit)


For more details on how to install and use the solution please visit the Codeplex site. Thanks for all the feedback received, enjoy!

Monday, March 2, 2015

Should I register my plugins in Sandbox or no isolation?

As you are probably aware CRM plugins and custom workflow activities can either execute in isolation (sandbox) or without isolation. This post explores when you have that choice and what good practices you should consider for making that choice.

The CRM Sandbox is a feature released in CRM 2011 whose objective was to address one problem: How can we trust custom code to execute in a server that hosts multiple customers? Certainly, the core scenario is for CRM Online in which you have different tenants (customers) sharing the same infrastructure. You can imagine Microsoft would not allow any random .Net code to execute on their data centers without some controls. The same would apply if you are a partner who hosts tenants for a third party: You have to be careful about the code that executed in your infrastructure.

So in order to execute someone else’s code in a secure manner you must make sure the code executes in a sandbox environment which has certain limitations. For example:

1. Code is running in partial trust. This prevents operations like accessing the local file system, registry, event log and many more. This is managed by .Net Code Access Security feature, you can read more here: . Note that by default sandbox plugins can sent HTTP or HTTPS messages to external endpoints, such as custom web services or Windows Azure. It is possible to overwrite this default to prevent sandbox plugins from calling external endpoints (other than localhost). You can overwrite this default in the registry (HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\MSCRM\SandboxWorkerOutboundUriPattern). More info on this here.

2. Throttling is applied to your plugin if it exceeds a given CPU/memory threshold. You can imagine this being useful to prevent someone’s code hijacking all the server resources. The process which executes your plugin can be killed if your plugin exceeds certain thresholds or becomes unresponsive.

3. Certain statistics are calculated for your sandbox plugins. These statistics typically include data such as how many times a given plugin type has crashed and what is the average execution time. These statistics are useful for providing throttling to someone else’s plugins.

The exact mechanisms used by Sandbox are not  all fully documented, this is perhaps because it reduces the risk of vulnerability of someone exploiting sandbox in CRM Online. However, what we do know is that sandbox plugins are executed in a special sandbox worker process that supports partial trust. Each CRM organization has its own sandbox worker process. Therefore, if someone’s malicious code results in the process getting killed, nobody else will be affected. Furthermore, we know there is a sandbox host process which you can see running as one of the windows services of CRM.

If you are working on CRM Online you have no choice and you must register all your plugins and custom workflow activities in isolation (sandbox mode) because of security reasons. However, if you are on premises, you have the choice, so which one should you pick?

You might hear that it is a good practice to always register on sandbox because if is more secure and you get free plugin statistics. Additionally, your solution will work for an Online customer or an on-premises customer without any modifications. While it is true that if you are an ISV and you sell solutions in the marketplace, it makes it easier to manage if your plugins are always in sandbox so you don’t need to release SKU-specific solutions (one solution for Online customers and another for on-premise customers). However, I disagree with anyone telling me that registering in sandbox is a good practice. It makes sense for the ISV scenario, but if you are simply writing a custom solution for your on-premise customer, you are not really getting any advantage by using sandbox. On the contrary, you are losing valuable system performance by doing so (more on this later).

So unless I have a real requirement to compute and regularly report my plugin statistics, I don’t see this feature adding much value to my on-premises customer. If your plugins crash or fail then you have a bigger problem to solve than looking at statistics. Perhaps measuring how long your plugins take to execute can be useful in some scenarios, but if you have never looked at these (or you don’t know of the existence of these statistics) then it is a good indication that you don’t need them. You might find them valuable in some cases, but at the end of the day, you are losing so many points for the slow performance of sandbox plugins that you really need to think twice what you value more: Some statistics you might never look at or a faster performance of your CRM?

The reason I talk about performance is because sandbox plugins are slower to execute that non-isolated plugins. I would have loved to have some data to show you but this is an easy test you can do, or I might do it in the future. Instead, I will explain why you can expect slower performance when using sandbox. Whenever CRM needs to execute a sandbox plugin, the process executing the operation (either IIS w3wp.exe or CrmAsyncService.exe) will need to do the following:

1. Temporarily suspend the current transaction (there is a time limit for this suspension)

2. Serialize the entire exeuction context of the current transaction and send a request to the sanbox host process to execute a given plugin on the current context

3. The sandbox host process will then deserialize this information and will then again serialize the information in order to delegate to the assigned “worker” process to execute the plugin.

4. The worker process then deserializes the current execution context and runs the plugin code in partial trust.

5. The result is then sent back to the sandbox host process.

6. The result is then sent back to the original process who requested a sandbox plugin (w3wp.exe or CrmAsyncService.exe).

Below are some diagrams to help you understand the flow in each scenario:






As you can see, this requires a lot of chatter between various processes and app domains, which is not exactly something you get for free. Additionally more room for errors in your infrastructure that can occur when you require multiple processes to be synchronized. Note that your plugin always executes in the sandbox worker process when registered in sandbox. If your plugin is not registered in isolation then it will be executed directly by the w3wp.exe process or the CrmAsynService.exe process (for async plugins and async workflow activities).

Therefore my conclusion is that you should not unnecessarily sacrifice system performance by registering your plugins in sandbox unless you have a good reason to. If you work with CRM Online you have no choice and if you are an ISV it might make your life easier to use sandbox; but I suggest you always make non-sandbox (no isolation) your default choice unless you have specific requirements or limitations that force you to register your plugin in sandbox mode. Additionally, you need to consider the limitations of sandbox plugins. For example, if you desire your plugin to use the event log or a local file system or shared drive, you cannot register your plugin in sandbox.

Friday, January 30, 2015

How to send an email via workflow without using the Send Email step (use create step instead)

You might already wonder: why the heck wouldn’t I just use the awesome “Send Email” step available in the workflow designer?! This post explains why but how to easily work around it.

We recently came across a scenario in which we wanted to automate some emails via workflows. However, we also needed to add some attachments to those emails based on some logic. Unfortunately in the current CRM workflow designer you cannot have any logic to add dynamic attachments to your email when you use the “Send Email” step of a workflow definition. However, we knew we could develop a simple custom workflow activity that could add attachments to the email. The problem was: If you use a Send Email step, the email is sent immediately and you have no way modify the email with attachments.

So the easy work-around is to first use the “Create” step in the workflow designer to first create an instance of the email an populate all the fields as desired. Then you can add additional steps that take actions on those emails, for example, a custom step that adds attachments to it. Because the records from the “Create” steps are available in all subsequent steps in the workflow, you can always reference it back and update it as many times as you want.

Once you have manipulated your email and added the relevant attachments with a custom step you are finally ready to send it, and this is the tricky part. How can you add a step to send an existing draft email? Well, certainly you could develop another custom workflow activity to do so. However, the solutions turns out to be much easier: All you need to do is set the status of the email to “Pending Send” and set the No. of Delivery Attempts to 0. Once you do this, then the email router will be ready to deliver the email (of course this is assuming you have configured your email router!).

So with a simple “Change Status” step and a subsequent “Update” step you can force the workflow to send the email you created!