Got RAM?

I remember my first experience with .Net Framework and C# after years of MFC/COM/DCOM torture with lovely C++. Whoa, I don't have to track every reference and resource allocation? That GC thing rules! No more memory leaks! Right...

Let's say, you're tasked with integrating Sitecore Commerce 9 and some PIM. It totally makes sense to use the Sitecore Commerce catalog system, so your integration would imply importing catalog structure and contents from PIM. While API capabilities of PIMs may vary, pretty much every type of PIM supports file exports of catalog. The native Sitecore Commerce catalog import/export is still in its infancy (no support for partial, incremental, or selective operations, not to mention lack of converters into its pretty unorthodox format, as well as inability to import product assets), so your only option is to build a custom catalog import plugin. The code for creating catalogs, categories and sellable items is relatively well-known and somewhat documented, so there should not be any surprises. You create a minion that watches for new PIM exports and consumes them. Everything is very straightforward. You create pipelines for updating categories, products, variants, and corresponding relationships. In order to perform operations with entities, you need instances of commerce commands. IoC takes care of that for you with something like this:

	public ImportCategoriesBlock(CreateCategoryCommand createCategoryCommand)
	{
	this.createCategoryCommand = createCategoryCommand;
	}

view raw ImportCategoriesBlock.cs hosted with ❤ by GitHub

OK, now you can iterate through the new categories and use this command to create them in catalog. For example:

	foreach (var category in categories)
	{
	var createdCategory = await this.createCategoryCommand.Process(context.CommerceContext, catalogName, category.CategoryName, category.DisplayName, category.Description);
	}

view raw ImportCategoriesBlock.cs hosted with ❤ by GitHub

Easy! And it works just fine in tests. However, when you decide it to run against your full catalog with dozens of thousands categories, the Commerce Engine suddenly starts eating up gigabytes of your precious RAM. How could that be even happening? Moreover, memory gets eaten much faster if you use several commerce commands within the same pipeline block - for example, CreateCategoryCommand, AssociateCategoryToParentCommand, EditCategoryCommand, DeleteCategoryCommand, and DeleteRelationshipCommand for the full category synchronization package.

All commands in Sitecore Commerce 9 derive from the Sitecore.Commerce.Core.Commands.CommerceCommand base class, which in turn contains Models property:

	namespace Sitecore.Commerce.Core.Commands
	{
	public class CommerceCommand : ICommerceCommand
	{
	/*.....*/
	public List<Model> Models { get; set; }
	/*.....*/
	}
	}

view raw CommerceCommand.cs hosted with ❤ by GitHub

Every time a command is used, this property gets populated by CommandActivity with the full instances of the entity models that were affected by that command. This includes versions, localizations, and such entity types:

	namespace Sitecore.Commerce.Core
	{
	public class CommandActivity : Activity
	{
	/*...*/
	private static void Activity_ActivityComplete(object sender, EventArgs e)
	{
	/*...*/
	commandActivity.Command.Models.AddRange((IEnumerable<Model>) commandActivity.Context.GetModels());
	/*...*/
	}
	/*...*/
	}
	}

view raw CommandActivity.cs hosted with ❤ by GitHub

Eventually a poor command ends up with hundreds of thousands hefty models in its Models property. This is a common architecture approach - to collect data updates and write them in a single batch before the disposal of the command object thus reducing number of calls to the database. However, it does not exactly work with huge number of updated entities. I'd like to think that Sitecore is planning on implementing some kind of collection monitor that would drain the Models list once it reaches a certain size. For now (as of Commerce 9 Update 3) that is your responsibility. While it might look like a bad design decision from the performance point of view, instantiating a new command before each use made import considerably faster. Don't forget, you can always do it using the CommerceCommander:

var createCategoryCommand = this.commerceCommander.Command<CreateCategoryCommand>();

view raw ImportCategoriesBlock.cs hosted with ❤ by GitHub

Of course, you can improve performance even further by creating a new command after each 100th, 1000th, or 10000th use. Just watch your RAM...