Agenda
- Preparations
- Configuration
- Module Configuration
- Performing Serialization Operations in CLI
- How to migrate from Unicorn to SCS
- Generating and Installing Item Packages
I used to use Unicorn and Sitecore TDS for past Sitecore solutions. Both tools are good but now, when one of my projects is about to finish, I expect to start with the new one soon and decided to have a look at new Sitecore Content Serialization that is available starting from Sitecore 10.
According to Sitecore:
Sitecore Content Serialization (SCS) is a system for serializing, sharing, and deploying content items, as well as keeping them in version control.
Basing on the description, I expect a new SCS tool to replace the tools I was using. So, let's have a look how good the SCS is.
Note: personally, I am not a big fun of TDS and prefer to use Unicorn. Thus, I will be comparing SCS with Unicorn only.
Note: all assumptions in this article are based on my investigations and might be wrong. I do not have experience with using the tool and might be incorrect in decisions I made.
To start my investigations I have installed a fresh Sitecore 10 version to my development PC and created very simple solution with three empty Foundation, Feature and Project projects. I will be using it to see how SCS can solve the difficulties I saw with my Sitecore solutions in the past.
Preparations
Install Sitecore Management Services
First of all I have figured out that I need to install Sitecore Management Services on my CM instance to start using Sitecore Content Serialization. This is just a regular Sitecore package that contains files only.
This package installs a service that allow to connect to Sitecore using Sitecore Command Line Interface or Sitecore for Visual Studio and execute serialization commands.
When we use Unicorn, we also need to copy some Unicorn specific files to Sitecore, so this looks familiar. In most cases, Unicorn specific files were part of the build artifacts and pushed to Sitecore during deployment.
For Sitecore Management Services I expect to use the same approach. To do this, I am going to unpack the package and keep it as part of my solution. My development deployment scripts will just copy files to build artifacts that will be moved to Sitecore CM instance at some point. This approach automate the package installation process and we do not need to ask all developers to install it manually.
BTW: the first thing I did was downloading the latest version of Sitecore Management Services (which was 3.0.0) and installed to Sitecore 10. As a result... Sitecore did not start and failed with an exception. After brief investigation, I have figured out that Sitecore 10 works with version 2.0.0 and version 3.0.0 is for Sitecore 10.1. It was a bit unclear from the download page... at least for me :)
Install Sitecore Command Line Interface
Good. The next step is to install Sitecore CLI. Installation details you can find here. Basically, it is just a few commands that needs to be executed in the root folder of your solution (in case you install it to the local project).
As a result, you will get a folder with name ".config" with only one file inside: "dotnet-tools.json". If you open the file, you can see the version of the CLI that is used. I had to change it to 2.0.0 since I was not sure that tool 3.0.0 will work with Sitecore 10 services.
Unicorn, in its turn, has an administration web page to do serialization commands or Powershell script to execute commands from Visual Studio tasks.
Both approaches used in Unicorn and SCS look fine to me. Good job!
Configuration
Before we start using serialization, we need to connect CLI with Sitecore instance. The process is described here. Basically, you need to use either interactive or non-interactive login. Again, I decided to use non-interactive one since this option is going to be used in deployment pipeline. After I did all necessary changes to Sitecore Identity Server and Sitecore CM I tried to execute login command as described in the documentation but got a message in Powershell. The message says that I need to execute "sitecore init" command first. Looks like I missed this step when travelling from one documentation page to another one.
"dotnet sitecore init" command needs to be executed in the root folder of your solution. After execution file "sitecore.json" will be created. This file contains configuration for the content serialization. More information about possible settings you can find here: general description, Item path length, Relative path, etc.
After review this file I noticed a few things:
1. The path to the module configuration does not correspond to my solution project
2. The path for serialization folder is located under the project while I used to see serialization folder on the same level as code folder that contains the project.
My spike project has the layout below:
/src/Feature/Feature Project/code - feature project code
/src/Feature/Feature Project/serialization - feature project serialization
/src/Foundation/Foundation Project/code - foundation project code
/src/Foundattion/Foundation Project/serialization - foundation project serialization
/src/Project/Project Project/code - solution project code
/src/Project/Project Project/serialization - solution project serialization
Thus, I changed modules property in sitecore.json file to the folowing:
"modules": [
"src/*/*/*/*.module.json"
],
"defaultModuleRelativeSerializationPath": "../serialization"
Seems fine but once I see serialization setting I have got two question.
Q1: Is it possible to have a single serialization folder for all modules? Sometimes you need to get all serialization files in one place and it much faster to copy if all serialization files are stored on the same level as solution file. In most of my projects, build script needs to search for all serialization files an copy it under the same root artifacts folder for further deployment. This operation was pretty time consuming and in some project there was a decision to change the default Unicorn serialization root by changing the "physicalRootPath" attribute of the "targetDataStore". The default value was:
<targetDataStore physicalRootPath="$(sourceFolder)\$(layer)\$(module)\serialization" useDataCache="false" singleInstance="true" />
A1: I expect to use the same trick with SCS but... I was not able to find a better way then specifying the full path to the serialization folder. Well, the answer to this question is Yes, it is possible but... are you sure that all developers will be using the same path to the solution and we can safely hardcode it? What about build agent?
I hope there is a way to specify the relative path basing on the solution and not project. Probably I just did not find it.
However, while trying to find an answer for Q1 I got another question.
Q2: is it possible to override settings in this file? I understand that all settings in this file must be the same for all developers, otherwise they we get conflicts with serialization files. However, in case there is no way to specify serialization path relative to solution and we have to use full path, I need a possibility to override the setting on developer PC.
A2: I was not able to find the answer to this question. Probably, we do not even need to do such things and it is good that we cannot override :).
Module Configuration
Ok, now we need to create configuration file for each project that have serialization in my solution. I failed to find a command in CLI that can create a default configuration file. Thus, I need to copy the example from here or here.
Note that suggested naming convention for this file is <name of module>.module.json
This means that just copy\paste will not work and you need to rename the file for every project. This might be fine. What is much worth here is that in the file, you have a few more places where you need to specify module name:
namespace - required - The namespace of the module
name - required - name of the folder that will keep the serialization
Hm.. there is a huge chance to do a typo or forget to update in case of using copy \ paste from the other project. Unicorn using variables \ tokens in configuration which make things much easier and you need to specify name of the module \ project only once.
I have seen a variables property in file sitecore.json but I am not sure how to use it since I failed to find any documentation regarding variables.
Apart the problem mentioned above, Sitecore has pretty rich rules configuration to control what is going to be serialized, what should be skipped and how to update the content if it already present in Sitecore. I was not able to find a situation that is covered by Unicorn but not Sitecore. More about rules you can read here.
However, I have a few questions that I was not able to find an answer for.
Q1: How to change the serialization behavior basing on environment? For example, I am developing a new feature an need some configuration item is present only on development environment but not on UAT and above. Unicorn is based on Sitecore configuration files and I can use roles to configure or patch the behavior. How do a similar thing with SCS?
A1: Well... You probably do not need the same thing here. The difference with Unicorn configuration and SCS configuration is that Unicorn related configuration is pushed to Sitecore and used when you do a Sync during deploy while SCS configuration is always part of the solution and never go to Sitecore. The delivery of items to Sitecore is different here - your build script produces an item package to be installed during deploy instead of working with serialization files. When installing package, you can specify which modules need to be installed or skipped. So, the answer to the question is: kind of yes but, instead of using configuration rules you need to have a separate development specific module in project layer that brings only development specific items. At least I would try this approach now.
Q1: How to change the behavior for the item installation? For example, I want the item to be overridden for non-production environment but created only when it does not exist on Prod environment. This requirement is valid for some configuration items that needs to have a different value for different environments.
A2: I did not find such a possibility except of using different environment specific project modules.
Performing Serialization Operations in CLI
- explain - Explains whether a content item path is included and why
- info - Shows serialization configuration information
- diff - Compares the content items of two Sitecore instances. Well, this is an interesting one but not sure I will use it often. I would like to have a command that does a dry run pull from Sitecore to file system and show me potential changes :)
- etc.
How to migrate from Unicorn to SCS
[/sitecore/content/Can be overriden] INCORRECT FILE PATH: ~/_page rendering components,[/sitecore/content/Can be overriden] found: ~\Can be overriden\8dcbf151-d484-4c3c-a68e-8f782cf9ae3e\_page rendering components.yml,[/sitecore/content/Can be overriden] expected: ~\Can be overriden\_page rendering components.yml[/sitecore/content/Can be overriden] > Fix will move to correct location
I followed the suggested approach and executed the command below:
dotnet sitecore ser validate --fix
This fixed the problem and moved the file to appropriate location!
Thus, it looks to be possible to migrate. The only question is how CLI behaves when hundreds of files needs to be moved.
Thumbs up to Sitecore!
Generating and Installing Item Packages
There are package operation commands in CLI. More about this here.
As I understand, Sitecore suggests to generate item package during the build pipeline and use package to deliver items instead of doing Sync from serialization like we did with Unicorn.
However, let's return to the questions I have in section "Module Configuration". If I need to have environment specific modules to specify environment specific behavior, I need to be able to either generate environment specific package or specify what modules needs to be ignored during package installation.
Right, Sitecore offers this behavior and include \ exclude modules for both commands: when generating and installing package. Nice!
However... I expected to specify top root project module during package generation and get all content it depends on (module configuration has references section where you can specify dependencies). Unfortunately, I got only content that belongs to the specified module in include regardless configuration in references property :( Hope this is just an issue that is going to be fixed.
Anyway, I was able to generate production package using exclude parameter and remove all non-production data. Good.
Q1: Where to use exclude \ include parameters? Is it better to do on package generation or during installation?
A1: Well, from one side, I do not want to spend the time on generating more than one package in build pipeline. This is not as fast as I would like to be with large amount of serialization files. At the same time, having different package for different environments minimizes a risk of getting non-production content on production environment. So, I would try to minimize a possibility to have more then one package and keep all environment specific configuration in configuration files instead of using content. However, if we do need to have different content, I would probably choose to have a separate packages - one for production content and another that have only environment specific content that needs to be installed after production one.
Q2: How to install item package in deployment pipeline?
A2: Good question... I do not know yet since I did not go so far :) I believe that we need to install CLI on deployment agent. Hope this is an easy task :)
Summary
In this post I described my impression from using new Sitecore Content Serialization that supports Sitecore 10+ version.
There are some questions and hidden areas that bubbled up during my spike. Probably I will get even more when I start using SCS with the real project. However, I must admit that Sitecore did a great job and new service looks very good.
I will definitely would like to try it with the next project!
