Planning for Cache in Sitecore

This is a follow-up to my Unofficial Sitecore Planning Checklist, where I try to consolidate as much information as I can find on Sitecore’s built in caching.

One of the easiest ways to improve a visitor’s perceived impression of any website visit is through a well-tuned cache. A cache which has been properly tuned will be able to server of requested content quicker, sometimes magnitudes faster than rebuilding the HTML for a page on each request.

Sophisticated caching scenarios can be configured through third-party tools and applications, but even for the most highly trafficked site properly tuning Sitecore’s different cache options should provide the desired performance gains. As site cache is configured and tuned, be aware the need for vertical or horizontal scaling may become necessary. (For details on scaling Sitecore checkout Scalability)

Determine the cache limits required

The general practice for Sitecore caching tuning is to turn all cache limits off via the following simple cache patch config:

<configuration xmlns:patch="">
      <setting name="Caching.DisableCacheSizeLimits">
    <patch:attribute name="value">true</patch:attribute>

With the cache limits now disabled, Sitecore will try to manage based on current traffic and available resources to cache as much as possible. By performing well structured load test or allowing the site to run in this manner, the team should monitor both server resource utilization as well as the Sitecore cache levels to determine areas that should be manually limited to maximize performance.

To monitor what Sitecore has cached you can use the Sitecore admin page, http://&lt; your domain >/Sitecore/admin/cache.aspx, or leverage the Sitecore Marketplace module Caching Manager,

Cache Manager Options

After data collection has occurred you are ready to begin to evaluate which caches need their limits adjusted from the default level. For more information on the different cache limits that can be set see Sitecore CMS 7.0 – 7.2 CMS Performance Tuning Guide.pdf and Sitecore CMS 6.6 or later Cache Configuration Reference.pdf.

Configuring HTML Cache of Components

The first level of caching to configure to is review each rendering that is created and set the appropriate HTML level caching on it.

Rendering Cache Option

To turn HTML cache on for the rendering, select the first option Cacheable. From here you can customize the how different variations of the control should be cached via the different ‘Vary By’ options. Finally, you can mark that all cache be cleared when the search index is updated via the ‘Clear on Index Update’ selection.

The following fuller description of each cache option is taken from Chapter 4: Output Caching in Sitecore CMS 7.0 or later Presentation Component Reference.pdf.

Clear on Index Update

The Clear on Index Update property controls whether or not a control clears its cache when an item that is associated with the control is updated in the index. This is useful for any control that uses the new Search API to populate its data sources. For example:

  • You have a control that returns all the products that are on special offer from the index.
  • The Clear on Index Update property for this control is set to True.
  • The price of each product is stored in the index.

If someone updates the price of one of the products on special offer, the Clear on Index Update property will the trigger the control to clear its cache because something has been updated in the index that is bound to the control.


The VaryByData property controls whether or not output caching varies based on the data source of the presentation component.

Set the VaryByData property:

  • To False for components that do not generate different output when used with different data sources.
  • To True for components that generate different output when used with different data sources.


The VaryByDevice property controls whether or not caching varies based on the name of the context device.

Set the VaryByDevice property:

  • To False for components that do not generate different output when used with different devices.
  • To True for components that generate different output when used with different devices.


The VaryByLogin property controls whether or not output caching varies based on whether or not the user has authenticated. For caching configuration involving the VaryByLogin property, the layout engine treats all anonymous users as a single authenticated user.

Set the VaryByLogin property:

  • To False for components that do not generate different output for authenticated than for unauthenticated visitors.
  • To True for components that generate different output for authenticated than for unauthenticated visitors.


The VaryByParm property controls whether or not output caching varies based on rendering parameters passed to the presentation component. Solutions built with earlier versions of Sitecore may have used the token VaryByParam instead of VaryByParm. Update any instances of VaryByParam to VaryByParm.

Developers set the VaryByParm property:

  • To False for components that do not generate different output when passed different rendering parameters.
  • To True for components that generate different output when passed different parameters.


The VaryByQueryString property controls whether or not output caching varies based on query string parameters passed in the URL. The VaryByParm property causes output caching to vary based on rendering parameter values passed by the developer. The VaryByQueryString property causes output caching to vary based on parameters passed in the URL query string.

Developers set the VaryByQueryString property:

  • To True for components that generate different output when supplied different query string parameters.
  • To False for components that do not generate different output when supplied different query string parameters.


The VaryByUser property controls whether or not output caching varies by the domain and username of the context user.

Developers set the VaryByUser property:

  • To False for components that do not generate different output for different users.
  • To True for components that generate different output for different users, when the number of active users between publishing operations is relatively small. The VaryByUser treats all anonymous users as a single authenticated user.

To avoid excess memory consumption, avoid using the VaryByUser property except in solutions with relatively small numbers of users, or monitor cache utilization closely. The VaryByLogin property causes Sitecore to generate different output depending based on whether or not a user has authenticated, differentiating anonymous users from authenticated users. The VaryByUser property causes Sitecore to generate different output for each user

Other Caches

Sitecore contains a number of other caches that help to lower the number of calls required to the database for item information and indexes. These are best described in a post from the Sitecore Tactics blog entitled How Sitecore caching work.

Additional Helpful references

One Final Note

Starting in Sitecore 8.2, the Cache APIs where refactored, creating breaking changes to any custom cache providers used in previous versions. Be sure to check out the change list,



How to plan for performance

Recently I had the opportunity to sit through a series of management courses. One of the main focuses was how to provide an environment to help motivate your team members and increase performance. In parallel to these courses I was working with a client to define the steps required to properly secure and architect a scalable Sitecore installation. After some extensive research and reading, followed by re-reading I’ve tried to compile the ‘Unofficial Sitecore Planning Checklist’.

As Sitecore has matured as a product so has the method of documenting the product, sadly it seems we continue to be in a slow shift to easily consumable and discoverable documentation, it reminds me of the early days of my SharePoint work, where the community documented better the product team. Because of this positive growing pain, you may end up on really old blog posts as well as some ‘ancient’ PDF documentation, which is still plenty applicable. Finally, this checklist is a starting point as documentation and feature sets continue to change and growth with the product always check for the latest before finalizing your organizations plans.

Some Basics

Begin with Sitecore Experience Platform 8.0 the ability to scale both horizontally and vertically was introduced through flexibility to place different Sitecore components on individual or combined servers.

The key components to be accounted for during server architecture is

  • Content delivery (CD) server (including personalization)
    • the touch point to the visitors, HTML, personalization, etc..flow from here
  • Content management (CM) server
    • Central nerve to the entire site, authoring and reporting occur here
  • Content databases (SQL)
    • SQL databases that houses your content
  • Session state server
    • This is the method you choose to track contacts while the actively visit the site.
    • Listed as an actual server this can take the form of a server, SQL database, MongoDB collection, or just ran in memory.
  • Collection database (MongoDB)
    • Storehouse for xAnalytics as data is collected
  • Processing server
    • Performs the heavy lifting job number crunching
  • Reporting database (SQL)
    • Just a database that all the pretty tables and charts come from
  • Reporting service
    • Service that moves data from SQL to those pretty tables and charts
  • Email Experience Manager Dispatch database (SQL)
    • Finally a clearly named item (database used to manage the queueing and sending of emails)
  • Email Experience Manager Dispatch service
    • Another straight named item (API that the servers use to perform the actual sending, dispatching, of emails)

You’ll notice there are a number of SQL databases that are required to support the system. These can all run on the same SQL instance or cluster without issue.

If you plan to place any of the service components on individual servers, you will need be properly licensed by Sitecore for an additional server install as they all require an installation of Sitecore just like a content delivery or management server.

Checkpoint 1: CM and CD Setup

  • CD Server
    • Bare minimum requirements: 16GB RAM, 4 cores resulting in 8 threads
    • Things to watch out for
      • Site must process heavy business or search logic making it computationally heavy CPU increase may be needed
      • Caching strategy may require additional RAM to support better cache retrieval of HTML and images
      • In-Proc Session State will be store to RAM, in which case a high traffic site is going to need more RAM
  • CM Server
    • Bare minimum: 16GB RAM, 4 cores resulting in 8 threads
    • Things to watch out for
      • As increase of con-current authors editing increases additional RAM becomes required to provide the better experience (this number may be around 10 to 15)
      • In a standard installation the CM server performs the additional duties of processing server, reporting service, and EXM Dispatch manager
        • All these will impact the performance of the CM requiring both additional CPU (helps with processing) to additional RAM (helpful for large EXM dispatches as each email is built in memory before being sent.)

Checkpoint 2: Experience Database (xDB)

The Experience Database commonly referred to as the ‘xDB’, is based on the NoSQL solution MongoDB. The xDB is the primary storage for all analytics information and the registry of contacts and engagement automation states. Both Content Delivery and Content Management servers talk to the xDB performing read and write actions against it.

Because data is written and read at high rates from the xDB as visitor’s sessions start and end requires a proper amount of RAM and high speed disk, such as SSD, to maximize performance.

Using MongoDB as your collection database, you should install plenty of RAM and use SSD drives. Sharding can also improve performance significantly. Read the documentation on the MongoDB website to learn about the MongoDB architecture, replication, sharding, and configuration options. (taken from xDB Hardware Guidelines)

Depending on the version of Sitecore running there are official supported MongoDB versions. This does not mean a newer/older version will not work with your version of Sitecore, but Paragon nor Sitecore can guarantee the behavior.

Sitecore XP 7.5 series Sitecore XP 8.0 Initial Release to Update 4 Sitecore XP 8.0 Update 5 and later Sitecore XP 8.1 and later Sitecore XP 8.2 and later
Mongo 2.6 mmapv1 Yes Yes Yes Yes Yes
Mongo 3.0 mmapv1 No No Yes Yes Yes
Mongo 3.0 Wired Tiger No No Yes Yes Yes
Mongo 3.2.1 mmapv1 No No No No Yes
Mongo 3.2.1 Wired Tiger No No No No Yes
Mongo 3.2.1 Enterprise with data-at-rest encryption (Wired Tiger only) No No No No Yes

For the latest supported version table see Sitecore’s documentation:

Sitecore recommends that MongoDB is setup in a replication architecture with sharding enabled. The minimum configuration is to run two full capacity MongoDB servers for data and a third lower capacity server to act as the arbiter. This third server would not need to handle any data in the basic configuration.

MongoDB can run on either Windows or Linux/Unix servers, as long as the proper version of MongoDB is used the Sitecore application does not care.

Additional Helpful references

Multilist with Search Secret Sauce

Discover the Multilist with Search Formula

I find myself wanting to leverage the Multilist with Search field often when architecting a site. The ability to provide the users multiple pages of information to pick from while having the ability to perform a wildcard search makes it an excellent option for assigning taxonomy to a page or item. As many the benefits are for the editor, the headaches trying to recall the syntax to setup the source is always a pain for me. I spend an hour or so digging through old code snippets, notes, and google looking for the right syntax formula.

But that is to be no more, as this post will be my notes for future setup and hopefully yours too.

The Setup

Let’s say we have a product taxonomy which is made up of a series of nested folders for organizational purposes and a product can be assigned one or more. If we are building out taxonomy assignments for some products for our Coffeeshop, we may have list of items in the content tree, such as

Product Category Taxonomy

Our product template will be fairly simple and contain the following fields

Product Template

The field we are focused on is Product Taxonomy and how to properly point it at our taxonomy list as we want to allow the editor to pick one or more categories.

Gut Instinct

Your initial gut instinct, or at least mine, is to begin wiring up Product Taxonomy with DataSource= to set the pick list like any of the other pick types, so our value would be


Which ends up producing a list of everything. (If you try and get fancy and use the item ID you’ll end up with the same results.)
DataSource source

Start Search Location

The value we need to use is StartSearchLocation, which always points to a single item whose chidren and grand-children and great-*-grandchildren will be filterd and displayed. My gut feel would use a path like


And this to sadly ends with a list of everything in the content tree. The natural progression of trial and error means to use the ID, so we have



Template Filter

Feeling a little better now, but it includes the sub-folders which have been setup that we want to not allow for selection. To limit what is available for selection we can apply a second parameter appended via an ampersand (‘&’) to the source value. The addition is TemplateFilter, which accepts a list of pipe delimited (‘|’) template IDs. In this sample we want all Basic Setting items (7F289750-AA0F-49DB-B479-F7D4646061DB) and CSS Class items (B28694DA-2FE5-478E-A64F-AA918BA53796)


Template Filter

Page Size

If for some reason the default page size of 20 items doesn’t fit your editor’s needs, then there is an additional parameter that can be added PageSize which will limit the number of items shown. In this sample we only want to show 3 items per page.




In instances when you need to provide a very finely tuned list of items to pick for the editor, you can use the Filter parameter which supports simple to very complex Lucene queries. When using the filter, you must include a plus or minus sign with the field indicating the appropriate inclusion (plus sign) or exclusion (minus sign) you wish to achieve. In this example we want to only show the taxonomy items that include the sub-string ‘caff’


Filter with Lucene

Sitecore Query

Finally, for those who have really complex needs we can perform our filter via a traditional Sitecore query. Just by assigning it to the StartSearchLocation value. When using this method, note that you need to be sure to convert ‘=’ to ‘->’ to have them properly translated by the system. StartSearchLocation always needs to point to a single item of which the children will be selected and filtered for displaying.For this example, we only want the color items only, we could achieve this by filtering for specific items, or for the parent folder


Filter with a Sitecore Query


But wait there is some more

When working on building the perfect source for your field, and the field is always blank or you just keep getting the entire tree, be sure to check the log. Sitecore has provided some nice error logging to help you figure out what might need to be changed. These messages include

  • Content Editor – Multilist with search: Cannot find any item by query ‘{0}’ from the StartSearchLocation parameter. Location: ‘Source’ field of the ‘{1}’ field in the ‘{2}’ template
  • Content Editor – Multilist with search: Query ‘{0}’ from the StartSearchLocation parameter has incorrect format. Location: ‘Source’ field of the ‘{1}’ field in the ‘{2}’ template
  • Content Editor – Multilist with search: Cannot find any item by id ‘{0}’ from the StartSearchLocation parameter. Location: ‘Source’ field of the ‘{1}’ field in the ‘{2}’ template. ‘{3}’ will be used instead.
  • Content Editor – Multilist with search: Value ‘{0}’ from the StartSearchLocation parameter could not be treated as a valid GUID. Location: ‘Source’ field of the ‘{1}’ field in the ‘{2}’ template. ‘{3}’ will be used instead.

Finally, if you want to do some further exploring the full source to how the field parses the source into queries can be found with a little decompile magic of Sitecore.Buckets.dll, you’ll be looking for the SearchList class.


EXM 3.4 Support for a Local MTA

With the release of Sitecore Email Experience Manager (EXM) 3.4, I felt the need to revisit my original post on configuring EXM for usage with a local mail transfer agent (MTA), as there have been some changes in the steps.

Sending through Sitecore

The default sending is through a product called ‘Sitecore Email Cloud’ which is uses a new third party service Spark Post. The basic steps for setting this up post install are

  1. Navigate to -> App_Config \ Include \ EmailExperience on the file system
  2. Remove ‘.disabled’ from
    • Sitecore.EDS.Providers.SparkPost.config.disabled
    • Sitecore.EDS.Providers.SparkPost.Sync.config.disabled
    • Spark Post Config Files
  3. Talk with your Sitecore Representative to confirm purchase and licensing of Sitecore Email Cloud. A new screen exists in EXM that helps you track your current licensing state.

    EXM Subscription Administration

Custom SMTP Sending

In previous versions of EXM you could setup the usage of a ‘Local MTA’ via the configuration of a single flag as well as a few Sitecore Setting fields. In the latest version, we now have to turn on a self-contained config file, Sitecore.EDS.Providers.CustomSMTP.config.

  1. Navigate to -> App_Config \ Include \ EmailExperience on the file system
  2. Remove ‘.disabled’ from
      • Sitecore.EDS.Providers.CustomSMTP.config.disabled
      • Sitecore.EDS.Providers.CustomSMTP.Sync.config.disabled
  3. Custom SMTP Config Files
  4. Next, we need to configure our SMTP connection for our MTA inside of Sitecore.EDS.Providers.CustomSMTP.config. Full documentation of all settings can be found at Custom SMTP Config Settings
<smtpSettings type="Sitecore.EDS.Core.Net.Smtp.SmtpSettings, Sitecore.EDS.Core" 
    <proxySettings ref="exm/eds/proxySettings" />
  1. To connect to Amazon’s AWS as I showed in the older post MTA Configuration in Sitecore, check this post out for steps to registering and setting up AWS. The basic patch file would be
<configuration xmlns:patch="">
    type="Sitecore.EDS.Core.Net.Smtp.SmtpSettings, Sitecore.EDS.Core" 
          <userName>SMTP Username as provided by AWS, 
            NOT your Amazon account usernameuserName>
          <password>SMTP Password as provided by AWS, 
            NOT your Amazon account passwordpassword>
          <proxySettings ref="exm/eds/proxySettings" />

Download from GIST

Custom SMTP Sync

Sending of emails is great, but if you want to see data collected from you custom SMTP you must also make some updates to the Sitecore.EDS.Providers.CustomSMTP.Sync.config file. This file controls the POP3 connection to gather data.

To see what needs to be configured see The EXM configuration settings.


Using the Custom SMTP is great for use in development and even testing environments, but strong consideration needs to be taken into account before rolling it out into production. One of the challenges of using a Custom SMTP, is the lack of full analytics collection that can occur. This is mostly due to the limited amount of information that can be retrieved via POP3. The other major challenge when using Custom SMTP in production is the additional configuration and management you must perform for DNS processing, blacklisting, etc…

Be for committing one way or the other in production be sure to review the differences as laid out by Sitecore in The Sitecore Email Cloud compared to the Custom SMTP.


Cleaning up log messages for Geo IP Location

Starting with Sitecore 8.1 Geo IP lookup services come pre-installed and configured. All a site owner than need to do is log into the App Center and purchase the service. This is great from an implementer standpoint this is great as it’s one less configuration step that we have to take.

On the downside, if the site owners never purchase the lookup service the log will quickly clutter with ERROR messages.

ManagedPoolThread #12 16:01:15 ERROR Failed to perform GeoIp lookup for dd4795c0-1dca-ea8d-93c4-06d7f7aa5063
Exception: System.Net.WebException
Message: The remote name could not be resolved: ‘’
Source: System
at System.Net.HttpWebRequest.GetResponse()
at Sitecore.CES.Client.WebClient.ExecuteRequest(String requestUri)
at Sitecore.CES.Client.ResourceConnector`1.Request(String endpoint, Object[] parameters)
at Sitecore.CES.Discovery.EndpointSource.GetEndpoint(String serviceName)
at Sitecore.CES.GeoIp.SitecoreProvider.GetInformationByIp(String ip)
at Sitecore.Analytics.Lookups.GeoIpManager.GetDataFromLookupProvider(GeoIpHandle geoIpHandle)

The Fix

The fix isn’t hard; all it requires is a simple patch config to disable the lookup service.

<configuration xmlns:patch="">
            <setting name="Analytics.PerformLookup">
                <patch:attribute name="value">false</patch:attribute>

For the full details on the lookup service checkout the full documentation at