Thursday, 2 April 2015

Breaking changes: Introduction

Since I am working in Sustained Engineering team, one of my activities in the company is working on Updates.

Update in our company must improve the quality of the specific major version of the product by resolving a number of associated issues.The update release should comply to a number of requirements. Some of them (that I personally consider as very important) are:

  • Must improve the overall quality of the product by fixing issues received from our customers and internal teams. At the same time regressions are not allowed and must be fixed before the release.
  • Must not contain breaking changes, thus customer should not rebuild (recompile) their solutions and\or workarounds after upgrading to the Update.
  • Must be backward compatible, thus all customer applications and features should work without any changes.

The main idea behind these requirements is to simplify upgrade procedure by minimizing the number of changes from customer side.

In this post I would like to share some thoughts about one of the requirement we have - breaking changes.

Let's start from the definition:

breaking change - a change in one part of a software system that potentially causes other components to fail; occurs most often in shared libraries of code used by multiple applications.

As you can see from the definition it is not clear of what exactly can be considered as breaking. It can be any change that could break existing solution. Personally I differentiate changes by categories.

Note: it could vary from one project to another. Current categorization is project specific.

Categories of breaking changes:

1. Breaking Changes in API

API change - a change in the publicly visible definition of a type, including any of its public members. This includes changing type and member names, changing base type of a type, adding/removing interfaces from list of implemented interfaces of a type, adding/removing members (including overloads), changing member visibility, renaming method and type parameters, adding default values for method parameters, adding/removing attributes on types and members, and adding/removing generic type parameters on types and members, etc. This does not include any changes in member bodies, or any changes to private members (i.e. we do not take into account Reflection).

Source: http://stackoverflow.com/questions/1456785/a-definitive-guide-to-api-breaking-changes-in-net

Check for the most of the breaking API changes can be automated. For example, you can configure you CI to execute LibCheck tool on every build and send a report in case any breaking changes detected.

However there are some changes in API that could break the API but not detected by the tool. In some sources this is called binary level break - an API change that results in client assemblies compiled against older version of the API potentially not loading with the new version. Example: changing method signature, even if it allows to be called in the same way as before (ie: void to return type / parameter default values overloads).

There are more types of breaking changes in API (changes in behavior, semantic, changes in abstract classes that just throw an exception when implementation is missing, etc.).

Note: I am going to have a separate post, with various examples, related to breaking changes in API.

The strategy is our company is to use a combination of code review, the tool that could detect the breaking changes and running unit and integration tests on CI. The better coverage you have in your tests the higher chance to catch breaking API change before the release.

2. Breaking Changes in UI

In relation to UI we are trying to avoid of making any changes here. Only minor fixes with styles are possible.

Such approach is related to several things:

  • Any changes in UI requires review and updating documentation (at least we have to regenerate screenshots).
  • Customers might use own versions of existing pages or dialogs with some modifications. Usually this is done by copying existing pages and inheriting form existing classes. In case we remove any controls or move them to a different place this might break their customization.

Changes we usually allow to do in updates:

  • Make changes in styling and sizes.
  • Move controls inside the container, however we should avoid of changing the control order.
  • In some situations adding new controls might be Ok (depends on situation).

To say the truth any changes in UI should be analyzed carefully.

Possible way to track changes in UI:

  • Analysis of changes before fix.
  • Code Review.
  • Automated UI testing.
  • Manual UI testing.

3. Breaking Changes in Configuration

The general rule in relation to changes in configuration is "Do not change the configuration."

The changes in configuration are almost always done manually and bring additional headache for the customers who are doing an upgrade.

In case you do need to make changes in configuration files, the only allowed changes are:

  • Changes in comments :).
  • Adding new nodes (preferably at the end of the node set).
  • Adding new attributes (preferably at the end of the node set).

Any other changes, like changing the order of the nodes, renaming nodes, changing node or attribute values, etc., will change the node xPath which means that this can potentially break the custom logic or custom patch files.

The check for changes in configuration can be automated by comparing with configuration from previous version.

Another good indicator for breaking changes in configuration is upgrade procedure. In two words - updating the configuration files during the upgrading between Updates should be optional, everything must be working at least not worth as it was before the upgrade. If you see any exceptions related to configuration after the upgrade this is an indicator that something has been done in a wrong way.

The rule: situation when necessary configuration is missing is always possible. You should have a plan "B" and use fall-backs and default values.

4. Breaking Changes in Database schema

For the changes in database schema we should use the same rule to the one we have for changes in configuration - "Avoid of making any changes in schema for minor updates".

General requirements are:

  • New API should be able to work with old schema, or in a different words - updating the schema in minor updates is not mandatory.
  • Applying new schema must never break or even modify existing customer data.

Changes in indexes, stored procedures are also considered as breaking since they could already have customization.

There are some tools that could be used for automated check for changes in database. One of them called "DBComparer" but there might be a lot of other that could be much better :).

5. Breaking Changes in data, stored in database.

It is very difficult to say anything concrete here since guidelines might depend on the data you have in database. For example: in case you have any template data, that could be used by your customers to create own specific content, you should ensure that once you change your template you will not break all content created by customer. You should always think about custom data once decide to change anything in your data. New API should always be compatible with data from previous version.

Summary

Please, do not forget about customers when making changes to your system. Tracking of breaking changes is not an easy thing but your customers are really appreciate this. In case you do have to introduce breaking changes, please make it visible to a customer (highlight in Release Notes, put a warning\erorr to a log, show message, etc.) and ensure the instructions for further steps are clear and easy.

Try to automate the process for identifying breaking changes since fixing them at the late stage is very difficult or even impossible at all.


No comments:

Post a Comment

Sitecore Content Serialization - first look

Agenda Preparations Configuration Module Configuration Performing Serialization Operations in CLI How to migrate from Unicorn to SCS Generat...