Some template sections or fields are missing after using Update Data Templates commerce command in Sitecore 9.0.1

Recently I have been involved in the project based on Sitecore 9.0.1 + Sitecore SXA + Sitecore Commerce.

This was really a great experience to be and allow to look at some things from a different angle :)

In this post, I would like to share some problem I faced when extending the schema for the Sellable Item in commerce.
As mentioned above we needed to extend the default fields we can see for a sellable item in Commerce interfaces. This was not the easiest task but we managed to do this with the KB article (big thanks to everybody who was publishing it). After we added a new section we had to regenerate Commerce templates in Sitecore backend to see the new section in content items.

Problem
After we updated Sitecore Commerce templates using "Update Data Templates" commands in "Commerce" ribbon tab we figured out that some sections that were present before are missing now. The one that we noticed was section "Images". What is even more interesting was the fact that this problem appeared only in a few developers only.

Investigation and solution
After some investigations, we have found that is it "CatalogTemplateGenerator" who is responsible for template generation. The problem was related to that fact that it generates templates basing on the first available object (e.g. Category templates are generated based on the first found Category commerce entity, Sellable Item templates are generated basing on the first available sellable item commerce entity etc.). In our case, it found the same commerce entity for all developers but some developers did have images assigned to the sellable item entity in their commerce database but others - not.
The problem is happening to be jing of specific since it will not be reproduced for the bost of the sections and fields. But images field is a bit special since, on the commerce side, it is implemented as a list control. In case it has no value the information about images is not added to the entity json returned from commerce server.

To fix this problem we had to override default commerce command with an own one and use our own implementation of template generator. Unfortunately, we were not able to override implementation from the standard implementation since all methods in "CatalogTemplateGenerator" class are private and non-virtual. Thus, we had to copy-paste implementation using reflector and change the implementation of "EnsureTemplateFields" method from (taken from dotPeak):

private void EnsureTemplateFields(TemplateItem templateItem, JToken view, string section = "Content")
    {
      EntityView entityView = view.ToObject<EntityView>();
      if (entityView.Properties.Any<ViewProperty>())
      {
        foreach (ViewProperty property in (Collection<ViewProperty>) entityView.Properties)
        {
          TemplateFieldItem templateFieldItem = templateItem.GetField(property.Name) ?? templateItem.AddField(property.Name, section);
          using (new EditContext(templateFieldItem.InnerItem))
          {
            templateFieldItem.Title = property.DisplayName;
            templateFieldItem.InnerItem.Appearance.ReadOnly = true;
            templateFieldItem.InnerItem.Appearance.Hidden = property.IsHidden;
            templateFieldItem.InnerItem[TemplateFieldIDs.Shared] = this.IsFieldLocalizable(view, property.Name) ? "0" : "1";
            templateFieldItem.Type = !(property.OriginalType == typeof (bool).ToString()) ? (property.OriginalType == typeof (double).ToString() || property.OriginalType == typeof (int).ToString() ? "Number" : "Single-Line Text") : "Checkbox";
          }
        }
      }
      else
      {
        if (((IEnumerable<string>) this._blacklist).Contains<string>(entityView.Name) || !entityView.ChildViews.Any<Model>() || view[(object) "ChildViews"].FirstOrDefault<JToken>() == null)
          return;
        TemplateFieldItem templateFieldItem = templateItem.GetField(entityView.Name) ?? templateItem.AddField(entityView.Name, section);
        using (new EditContext(templateFieldItem.InnerItem))
        {
          templateFieldItem.Title = entityView.DisplayName;
          templateFieldItem.InnerItem.Appearance.ReadOnly = true;
          templateFieldItem.InnerItem.Appearance.Hidden = false;
          templateFieldItem.InnerItem[TemplateFieldIDs.Shared] = this.IsFieldLocalizable(view, entityView.Name) ? "0" : "1";
          templateFieldItem.Type = "Treelist";
        }
      }
    }

to something like this:

protected virtual void EnsureTemplateFields(TemplateItem templateItem, JToken view, string section = "Content")
        {
            EntityView entityView = view.ToObject<EntityView>();
            if (entityView.Properties.Any<ViewProperty>())
            {
                foreach (ViewProperty property in entityView.Properties)
                {
                    TemplateFieldItem templateFieldItem =
                        templateItem.GetField(property.Name) ?? templateItem.AddField(property.Name, section);
                    using (new EditContext(templateFieldItem.InnerItem))
                    {
                        templateFieldItem.Title = property.DisplayName;
                        templateFieldItem.InnerItem.Appearance.ReadOnly = true;
                        templateFieldItem.InnerItem.Appearance.Hidden = property.IsHidden;
                        templateFieldItem.InnerItem[TemplateFieldIDs.Shared] =
                            this.IsFieldLocalizable(view, property.Name) ? "0" : "1";
                        templateFieldItem.Type = GetPropertyType(property);
                    }
                }
            }
            else
            {
                if ((this._blacklist.Contains<string>(entityView.Name) ||
                    !entityView.ChildViews.Any<Model>() || view[(object) "ChildViews"].FirstOrDefault<JToken>() == null) && !entityView.Name.Equals("images", StringComparison.OrdinalIgnoreCase))
                    return;
                TemplateFieldItem templateFieldItem =
                    templateItem.GetField(entityView.Name) ?? templateItem.AddField(entityView.Name, section);
                using (new EditContext(templateFieldItem.InnerItem))
                {
                    templateFieldItem.Title = entityView.DisplayName;
                    templateFieldItem.InnerItem.Appearance.ReadOnly = true;
                    templateFieldItem.InnerItem.Appearance.Hidden = false;
                    templateFieldItem.InnerItem[TemplateFieldIDs.Shared] =
                        this.IsFieldLocalizable(view, entityView.Name) ? "0" : "1";
                    templateFieldItem.Type = "Treelist";
                }
            }
        }

Also, since already had own generator we decided to update the logic that resolves the template fields type. This logic was noticed in the same method and looks like the one below:

templateFieldItem.Type = !(property.OriginalType == typeof (bool).ToString()) ? (property.OriginalType == typeof (double).ToString() || property.OriginalType == typeof (int).ToString() ? "Number" : "Single-Line Text") : "Checkbox";

As you can see only "Number", "Single-Line Text" and "Checkbox" fields types are supported + some special logic for lists. We decided to change the implementation a bit and add support for additional types - "html" and "memo". We have extracted this to a separate method:

protected virtual string GetPropertyType(ViewProperty property)
{
  string type = "Single-Line Text";
  if (property.OriginalType == typeof(bool).ToString())
  {
    type = "Checkbox";
  }
  else if (property.OriginalType == typeof(double).ToString() ||
                     property.OriginalType == typeof(int).ToString())
  {
    type = "Number";
  }
  else if (string.Equals(property.OriginalType, "html", StringComparison.OrdinalIgnoreCase))
  {
    type = "Rich Text";
  }
  else if (string.Equals(property.OriginalType, "memo", StringComparison.OrdinalIgnoreCase))
  {
    type = "Multi-Line Text";
  }
    return type;
}

After these changes, the problem with missing "images"  section has gone and fields started to look better in Content Editor.