Tuesday, July 21, 2015

How to cascade activate / deactivate (SetState) in CRM

CRM allows for an easy configuration of the cascade behavior for delete, assign, reparent, merge and share. But how can we cascade the status of a record to its children?

This is a problem that I find in many CRM implementations. A typical example is that you want to deactivate all contacts related to an account whenever the account is deactivated. Similarly, if an inactive account is re-activated you might want to re-activate its child records such as the child contacts.

The good news is that I have updated the MSCRMWorkflowUtilities to handle this for you without having to write any code (currently only supported for CRM 2015+).  Simply download and install the solution. Then you can configure any number of cascade behavior for the SetState operation.

You will first need to define a workflow based on the parent entity. In the example above, we would define a workflow on the account entity in order to cascade its status to the child contacts. Then you can insert conditions and as many cascade steps as you want. In my example, I will check if the account is active then activate all the child contacts and if the account is inactive then inactivate all the child contats. However, note that you could insert additional steps if for example you want to deactivate all child contacts and activities then you need to insert one step for each child entity.

image

 

For each “Cascade SetState” step you insert, you need to configure 3 parameters:

image

1. Child Entity Name: The schema name of the child entity to which you wand to cascade the SetState operation. In this case it will be “contact”. You can obtain the schema name from the entity definition:

image

2. Child Lookup Attribute To Parent: This is the schema name of the attribute (field) of the child entity that links to the parent entity via a lookup. In this case it is “parentcustomerid”. You can obtain this name from the child entity definition by finding the lookup attribute to the parent entity:

image

3. Target Child State Code: The target state/status to cascade to the child entity. For most cases “1” is for deactivate and “0” is to activate. Note some special entities like case and opportunity are not supported because they don’t use a SetState message. All custom entities are supported.

You can imagine that there can be a performance problem when you try to deactivate a record which has a large number of child records. Therefore, it is recommended that your workflow be asynchronous instead of synchronous. Have fun!

Monday, June 1, 2015

Scalability considerations for CRM / SharePoint integration

SharePoint is probably the most natural and easiest to implement solution for document management for CRM records. However, what happens when you have millions of records in CRM which might have documents?

I recently worked on fairly large document migration solution to be able to associate documents to CRM records (e.g. cases, opportunities). There were a few hundreds of thousands of files amounting to about 50GB of data. So here are a few things to consider



1. FOLDER HIERARCHY



With the out of the box integration you have the choice to structure your folders based on accounts/contacts or simply to create a new library for each entity at the root of the site. It might be more convenient to navigate SharePoint if you can start from an account and then find which are the related cases/opportunities rather than starting with a global list of cases and try to find which cases are related to a specific account. In general, this structure is more convenient, until you consider security segregations.

If you need to segregate SharePoint security in such a way that some users can access documents of a given entity type (e.g. cases) but not be able to access documents of another entity type (e.g. opportunities) then the account/contact hierarchy becomes a problem because you cannot set the security at the library level (all documents would be on the same library) and you would have to do some acrobatics with folder security and inheritance breaking which would be a nightmare to maintain. Instead, if you have each entity type have its own SharePoint library then you can easily grant/deny access to documents of a given entity type.

Nonetheless, remember that with the out of the box CRM/SharePoint integration, there is no security synchronization between CRM and SharePoint. You need to keep this in mind throughout your design. Think for example that a user might not be able to access any opportunities in CRM but the user (if malicious) can always find a way to see the documents associated to CRM opportunities by going to SharePoint (unless you block the user completely from the given SharePoint library).

Also consider that changing the folder hierarchy after would be a major data migration effort so you should really think about what hierarchy makes more sense you’re your situation and consider current or future security requirements.








2. NUMBER OF ITEMS PER FOLDER / LIBRARY

Imagine that you have 100K cases in your system and you have enabled SharePoint integration for case entity. And perhaps every case has at least one document. In this case, your case library/folder will contain 100K items flat on the same list. This goes well beyond what SharePoint recommendations are for scalability and performance. It is not recommended to go beyond 5000 items (even that is already quite high). Of course you can always implement some sort of archiving or use multiple SharePoint sites depending on some criteria so you split this load. Another [reasonably] easy solution is to further structure your folders by year, quarter or month (or all of these). This way you will not end up with 100K folders under the “cases” folder. Instead, the maximum number of folders you will have under a single folder will be the maximum amount of cases that can be opened in a given month/quarter which might be a more reasonable number.





In the example above, we have 2 new layers: Year and Month. These correspond to the date on which the case was opened. By adding these additional layers, we can now guarantee that there will be no folder with more than a few hundred sub-folders, since we know that we only open a few hundred cases per month.

The downside is that this cannot be done by simple configuration or out of the box integration. This structure would require that you register a plugin on create of case, which will create the document locations and the SharePoint folder for the case being created. Some other disadvantages of this approach is that:

- Every case will now have a folder, even if the case has no documents. This should not be a problem if you know that anyway all cases have documents

- You are creating the SharePoint folder at the same time as you create the case, instead of the OOB behavior which is to create the folder on demand the first time document library is accessed in CRM. This is not necessarily a bad thing though.

- If your plugin is sync and it fails (e.g. SharePoint is down) then it will prevent the creation of the case in CRM. If your plugin is async and it fails, then you need to have a way to recover and create the correct folder when the user tries to access document library for this record.





3. SIZE AND GROWTH OF YOUR SHAREPOINT SITE

For sites with large volume of documents, large document sizes or rapidly growing volumes, you might also need to consider how long you have until you start having a performance or limitation problem with things like the max size of your content database, max number of files in the library or simply the maximum size of your site. O365 also has some limitations that you need to review. If you identify that this could be a potential problem you should consider implementing some sort of archival solution which will allow you to keep a link between CRM records and SharePoint files while at the same time optimizing for current and mostly used records. I have heard of people simply changing the CRM site every now and then and creating a new site once the old site is getting too large. I guess any of these strategies would work as long as you have defined a process to scale your site and monitor the volumes regularly. The most important thing is to consider this during your design phase and have identified your approach to handle scalability, even if you might not implement a long term maintenance strategy right away.

Tuesday, April 7, 2015

CRM 2011 Workflow Utilities now available for CRM 2013 and 2015

By popular demand, I have upgraded the CRM 2011 workflow utilities and rebranded it to be compatible with CRM 2013 and CRM 2015.

You might be familiar with the CRM 2011 Workflow Utilities project that I released to Codeplex a few years ago (https://crm2011workflowutils.codeplex.com/). By popular demand I have released version 3.0 for CRM 2015 (both for Online and On Premises). You can download the CRM 2015 version here: http://mscrmworkflowutilities.codeplex.com/releases/view/613024

Note that if you are still on CRM 2011 or CRM 2013, you must continue to use the release version 2.0.2.0 available here: http://mscrmworkflowutilities.codeplex.com/releases/view/102880 which has 2 solutions, one for CRM Online and another for CRM OnPremises.

Additionally, I have rebranded the tool from “CRM 2011 Workflow Utilities” to “MSCRM Workflow Utilities” since it supports multiple CRM versions (and hopefully future versions too!).

Workflow utilities is a CRM solution that allows you to extend the CRM workflow design experience by providing the following custom workflow steps available directly from the CRM process designer:

  • Delete record
  • Share or “unshare” a record
  • Insert hyperlink to a CRM record
  • Qualify lead (convert to account/contact/opportunity)
  • Bulk activate / deactivate records (no record count limit)

 

For more details on how to install and use the solution please visit the Codeplex site. Thanks for all the feedback received, enjoy!

Monday, March 2, 2015

Should I register my plugins in Sandbox or no isolation?

As you are probably aware CRM plugins and custom workflow activities can either execute in isolation (sandbox) or without isolation. This post explores when you have that choice and what good practices you should consider for making that choice.

The CRM Sandbox is a feature released in CRM 2011 whose objective was to address one problem: How can we trust custom code to execute in a server that hosts multiple customers? Certainly, the core scenario is for CRM Online in which you have different tenants (customers) sharing the same infrastructure. You can imagine Microsoft would not allow any random .Net code to execute on their data centers without some controls. The same would apply if you are a partner who hosts tenants for a third party: You have to be careful about the code that executed in your infrastructure.

So in order to execute someone else’s code in a secure manner you must make sure the code executes in a sandbox environment which has certain limitations. For example:

1. Code is running in partial trust. This prevents operations like accessing the local file system, registry, event log and many more. This is managed by .Net Code Access Security feature, you can read more here: https://msdn.microsoft.com/en-us/library/930b76w0(v=vs.80).aspx . Note that by default sandbox plugins can sent HTTP or HTTPS messages to external endpoints, such as custom web services or Windows Azure. It is possible to overwrite this default to prevent sandbox plugins from calling external endpoints (other than localhost). You can overwrite this default in the registry (HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\MSCRM\SandboxWorkerOutboundUriPattern). More info on this here.

2. Throttling is applied to your plugin if it exceeds a given CPU/memory threshold. You can imagine this being useful to prevent someone’s code hijacking all the server resources. The process which executes your plugin can be killed if your plugin exceeds certain thresholds or becomes unresponsive.

3. Certain statistics are calculated for your sandbox plugins. These statistics typically include data such as how many times a given plugin type has crashed and what is the average execution time. These statistics are useful for providing throttling to someone else’s plugins.

The exact mechanisms used by Sandbox are not  all fully documented, this is perhaps because it reduces the risk of vulnerability of someone exploiting sandbox in CRM Online. However, what we do know is that sandbox plugins are executed in a special sandbox worker process that supports partial trust. Each CRM organization has its own sandbox worker process. Therefore, if someone’s malicious code results in the process getting killed, nobody else will be affected. Furthermore, we know there is a sandbox host process which you can see running as one of the windows services of CRM.

If you are working on CRM Online you have no choice and you must register all your plugins and custom workflow activities in isolation (sandbox mode) because of security reasons. However, if you are on premises, you have the choice, so which one should you pick?

You might hear that it is a good practice to always register on sandbox because if is more secure and you get free plugin statistics. Additionally, your solution will work for an Online customer or an on-premises customer without any modifications. While it is true that if you are an ISV and you sell solutions in the marketplace, it makes it easier to manage if your plugins are always in sandbox so you don’t need to release SKU-specific solutions (one solution for Online customers and another for on-premise customers). However, I disagree with anyone telling me that registering in sandbox is a good practice. It makes sense for the ISV scenario, but if you are simply writing a custom solution for your on-premise customer, you are not really getting any advantage by using sandbox. On the contrary, you are losing valuable system performance by doing so (more on this later).

So unless I have a real requirement to compute and regularly report my plugin statistics, I don’t see this feature adding much value to my on-premises customer. If your plugins crash or fail then you have a bigger problem to solve than looking at statistics. Perhaps measuring how long your plugins take to execute can be useful in some scenarios, but if you have never looked at these (or you don’t know of the existence of these statistics) then it is a good indication that you don’t need them. You might find them valuable in some cases, but at the end of the day, you are losing so many points for the slow performance of sandbox plugins that you really need to think twice what you value more: Some statistics you might never look at or a faster performance of your CRM?

The reason I talk about performance is because sandbox plugins are slower to execute that non-isolated plugins. I would have loved to have some data to show you but this is an easy test you can do, or I might do it in the future. Instead, I will explain why you can expect slower performance when using sandbox. Whenever CRM needs to execute a sandbox plugin, the process executing the operation (either IIS w3wp.exe or CrmAsyncService.exe) will need to do the following:

1. Temporarily suspend the current transaction (there is a time limit for this suspension)

2. Serialize the entire exeuction context of the current transaction and send a request to the sanbox host process to execute a given plugin on the current context

3. The sandbox host process will then deserialize this information and will then again serialize the information in order to delegate to the assigned “worker” process to execute the plugin.

4. The worker process then deserializes the current execution context and runs the plugin code in partial trust.

5. The result is then sent back to the sandbox host process.

6. The result is then sent back to the original process who requested a sandbox plugin (w3wp.exe or CrmAsyncService.exe).

Below are some diagrams to help you understand the flow in each scenario:

image

 

image

 

image

As you can see, this requires a lot of chatter between various processes and app domains, which is not exactly something you get for free. Additionally more room for errors in your infrastructure that can occur when you require multiple processes to be synchronized. Note that your plugin always executes in the sandbox worker process when registered in sandbox. If your plugin is not registered in isolation then it will be executed directly by the w3wp.exe process or the CrmAsynService.exe process (for async plugins and async workflow activities).

Therefore my conclusion is that you should not unnecessarily sacrifice system performance by registering your plugins in sandbox unless you have a good reason to. If you work with CRM Online you have no choice and if you are an ISV it might make your life easier to use sandbox; but I suggest you always make non-sandbox (no isolation) your default choice unless you have specific requirements or limitations that force you to register your plugin in sandbox mode. Additionally, you need to consider the limitations of sandbox plugins. For example, if you desire your plugin to use the event log or a local file system or shared drive, you cannot register your plugin in sandbox.

Friday, January 30, 2015

How to send an email via workflow without using the Send Email step (use create step instead)

You might already wonder: why the heck wouldn’t I just use the awesome “Send Email” step available in the workflow designer?! This post explains why but how to easily work around it.

We recently came across a scenario in which we wanted to automate some emails via workflows. However, we also needed to add some attachments to those emails based on some logic. Unfortunately in the current CRM workflow designer you cannot have any logic to add dynamic attachments to your email when you use the “Send Email” step of a workflow definition. However, we knew we could develop a simple custom workflow activity that could add attachments to the email. The problem was: If you use a Send Email step, the email is sent immediately and you have no way modify the email with attachments.

So the easy work-around is to first use the “Create” step in the workflow designer to first create an instance of the email an populate all the fields as desired. Then you can add additional steps that take actions on those emails, for example, a custom step that adds attachments to it. Because the records from the “Create” steps are available in all subsequent steps in the workflow, you can always reference it back and update it as many times as you want.

Once you have manipulated your email and added the relevant attachments with a custom step you are finally ready to send it, and this is the tricky part. How can you add a step to send an existing draft email? Well, certainly you could develop another custom workflow activity to do so. However, the solutions turns out to be much easier: All you need to do is set the status of the email to “Pending Send” and set the No. of Delivery Attempts to 0. Once you do this, then the email router will be ready to deliver the email (of course this is assuming you have configured your email router!).

So with a simple “Change Status” step and a subsequent “Update” step you can force the workflow to send the email you created!

image

image

Wednesday, November 19, 2014

Avoiding form reload when switching CRM forms based on a field

Quite often we define different forms for a given entity and we do a form switch based on a field rather than based on the security role of the user. The problem we observe with this approach is that there is often a double form load because when the wrong form loads then record re-loads using the appropriate form. This post offers a solution for avoiding the double form load in this scenario.

Say you have different types of opportunity which are identified by an “Opportunity Type” field and each opportunity type has its own form and BPF. The traditional approach would be to have a JavaScript on load of the form which will navigate to the appropriate form. The issue is that CRM web application loads by default the form that was last used by the given user. So if the user was looking at an opportunity of type “New Sale” and then opens an opportunity of type “New Service Contract” then the service contract opportunity will first load using the “new sale” form and after that form is loaded it will open the correct “service contract” form. The effect is bad user experience because the form loads twice so it doubles the form load time.

I have come across a solution that prevents this double form loading from happening which was suggested by a colleague. The solution is based on the premise that we can change which is the last viewed form for a given user on the fly when the user is retrieving a record. So you can intercept the retrieve operation from a plugin and then look at the type of opportunity that the user is retrieving and then update the user’s last viewed form to be the correct form for the opportunity type that is just getting retrieved. This way, the opportunity is always loaded in the correct form without a form switch.

The solution is a two-step process for the retrieve plugin:

1. In the pre-operation you must make sure to include the “opportunity type” field in the ColumnSet so that it is available later on for you to decide which is the appropriate form for the record.

2. In the post-operation you will have the “opportunity type” value and then you can update the user’s last viewed form to the correct form for the given opportunity type. To do so you simply need to modify the UserEntityUISettings entity for the given user and set the lastviewedformxml field.

Here is some sample code to use from your Retrieve plugin:

on Retrieve plugin
  1. var pluginContext = (IPluginExecutionContext)context;
  2. if (pluginContext.IsPreOperationStage())
  3. {
  4.     var columns = (ColumnSet)pluginContext.InputParameters["ColumnSet"];
  5.     if (!columns.Columns.Contains(OpportunityTypeAttributeName))
  6.         columns.AddColumn(OpportunityTypeAttributeName);
  7. }
  8. else if (pluginContext.IsPostOperationStage())
  9. {
  10.     var currentEntity = context.GetEntityFromContext();
  11.     if (currentEntity == null)
  12.         return;
  13.  
  14.     SetForm(currentEntity.ToEntity<Opportunity>(), service, context.UserId, tracingService);
  15. }

 

Set the correct form
  1. private void SetForm(Opportunity opp, IOrganizationService service, Guid userId)
  2. {
  3.     var query = new QueryExpression(UserEntityUISettings.EntityLogicalName);
  4.     query.Criteria.AddCondition("ownerid", ConditionOperator.Equal, userId);
  5.     query.Criteria.AddCondition("objecttypecode", ConditionOperator.Equal, Opportunity.EntityTypeCode);
  6.     query.ColumnSet = new ColumnSet("lastviewedformxml");
  7.     var settings = service.RetrieveMultiple(query).Entities;
  8.  
  9.     // Some users such as SYSTEM have no UserEntityUISettings, so skip.
  10.     if (settings == null || settings.Count != 1 || opp.pwc_OpportunityType == null) return;
  11.  
  12.     var setting = settings[0].ToEntity<UserEntityUISettings>();
  13.     string formToUse;
  14.     switch ((pwc_contractedengineservice)opp.pwc_OpportunityType.Value)
  15.     {
  16.         case pwc_contractedengineservice.NewSale:
  17.             formToUse = String.Format("<MRUForm><Form Type=\"Main\" Id=\"{0}\" /></MRUForm>", AdHocEngineServiceSaleFormId);
  18.             break;
  19.         case pwc_contractedengineservice.ServiceContract:
  20.             formToUse = String.Format("<MRUForm><Form Type=\"Main\" Id=\"{0}\" /></MRUForm>", ContractedEngineServiceSaleFormId);
  21.             break;
  22.         default:
  23.             return;
  24.     }
  25.     if (!formToUse.Equals(setting.LastViewedFormXml, StringComparison.InvariantCultureIgnoreCase))
  26.     {
  27.         // Only update if the last viewed form is not the one required for the given opportunity type
  28.         var s = new UserEntityUISettings { Id = setting.Id, LastViewedFormXml = formToUse };
  29.         service.Update(s);
  30.     }
  31. }

 

Regarding the supportability of this approach it seems to be a grey area. Technically you are doing operations via the SDK and using regular plugins which seems like a supported approach. The only problem is that the lastviewedformxml field of the UserEntityUISettings which is not documented in the SDK (although the entity seems valid for update).

A few things to consider is that if you are able to make this work with simple security roles then it would be simpler to configure the forms using security roles and you would not need this work-around. Consider as well whether you really need multiple forms or you can have other approaches such as JavaScript to hide/show sections depending on the type of record. I have posted some guidance regarding the different options available: http://gonzaloruizcrm.blogspot.ca/2014/07/different-entity-flavours-new-entity.html

Tuesday, October 14, 2014

Error when importing CRM solution 0x8004F658: The label {}, id: {} already exists. Supply unique labelid values.

I usually don’t post on CRM bugs but this one made me lose so much time that I figured I might share my experience.

We are in the middle of a CRM 2013 upgrade project and while transporting our main solution with all the entities we keep getting this error message: The label '{0}', id: '{1}' already exists. Supply unique labelid values while importing entity forms. I first thought that somehow we messed up the labels since we have a bilingual English and French system. However, after looking at the LocalizedLabel tables I could not find any duplicates of what the error message was talking about.

Then I decided to do a search in all the system forms in the target environment and check where the labelid occurred. I never really found a duplicate but I noiticed that in some cases the label already existed in the target environment in the formXML. This was as expected but to get over the problem I deleted the problematic forms in the target environment (since they will get overwritten by the new solution anyway). Some forms were the system forms so I could not really delete them, therefore I just stripped them down completely to remove all tabs, sections and fields as much as possible to make sure the custom labels would have been deleted.

Now I publish all the customizations and try importing again. Half of the failures are gone but I still get the same error message on some entity forms. I find it weird since the “duplicate” labels are nowhere to be found. I also validated that in my custoizations.xml the labelid was not duplicated, no, each GUID appeared only once.

With no time to call Microsoft support I had to find a work-around. So here’s what I do (not proud of the solution but it got me out of trouble). I open the customizations.xml file and locate the labelid which solution import is complaining about. And I simply generate a new GUID and replace it:

image

In some other cases there was not even a “labelid” attribute on the tab so I just added it with a new Guid. This happened in some tab, section and cell nodes, it was all over the place for some reason.

So after assigning a new Guid to all these sections I was able to import successfully. I never really had the time to go deeper into the root cause, but after talking to a colleague we suspect that the issue stems from the fact that we used the “Merge Forms” button that is available in CRM 2013 to help you upgrade your forms. The issue only occurred for the entities for which we used that feature so therefore our suspicion.

image

I will never be sure but I will try to avoid using that button until I have time to get back to the root cause analysis of this issue, lot of time wasted.