Refactor vs Replacement: a Journey

Journeys around the refactoring of application and system architectures are dangerous.  Every experienced developer probably has a horror story around an effort to evolve an application that has gone wrong.  Because of this, any consideration around the refactoring effort of an application or system needs to start with a frank evaluation as to whether or not that application has hit the end of its lifecycle and should be retired.  You should only consider a refactor if you are convinced that the return on investment for that refactor is higher than the return on a rewrite or replace. Even then, it should not be an automatic decision.

First, let us define a refactor.  The multiple ways in which this word is used can lead to confusion.  The first approach is best described by Martin Fowler, who defined it as “A change made to the internal structure of software to make it easier to understand and cheaper to modify without changing its observable behavior.”  That is not the appropriate definition in this case.  Instead, refactor in this case refers to a process that evolves the software through multiple iterations, with each iteration getting closer to the end state.

One of the common misconceptions of an application refactor is that the refactor will successfully reduce problematic areas.  However, studies have shown that refactoring has a significantly high chance of introducing new problem areas into the code (Cedrim et al., 2017).  This risk increases if there is other work going on within the application, thus an application that requires continuing feature or defect resolution work will be more likely to have a higher number of defects caused by the refactoring process (Ferriera et al., 2018). Thus, the analysis regarding refactoring an application has to contain both an evaluation of the refactor work and the integration work for the new functionality that may span both refactored and refactored work.  Unanticipated consequences when changing or refactoring applications is so common that many estimation processes try to consider them.  Unfortunately, however, these processes only try to allow for the time necessary to understand and remediate these consequences rather than trying to determine where these efforts will occur.

Analyzing an application for replacement

Understanding the replacement cost of an application is generally easier than analyzing the refactor cost because there tends to be a deeper understanding of the development process as it pertains to new software as opposed to altering software; it is much easier to predict side effects in a system as you build it from scratch.  This approach also allows for “correction of error” where previous design decisions may no longer be appropriate.  These “errors” could be the result of changes in technology, changes in business, or simply changes in the way that you apply software design patterns to the business need; it is not a judgement that the previous approach was wrong, just that it is not currently the most appropriate approach.

Understanding the cost of replacing an application then becomes a combination of new software development cost plus some level of opportunity cost.  Using opportunity cost is appropriate because the developers that are working on the replacement application are no longer able to work on the current, to be replaced, version of the application. Obviously, when considering an application for replacement, the size and breadth of scope of the application is an important driver.  This is because the more use cases that a system has to satisfy; the more expensive it will be to replace that system.  Lastly, the implementation approach will matter as well.  An organization that can support two applications running in parallel and phasing processing from one to the other will have less of an expense than will an organization that turns the old system off while turning on the new system.  That you can phase the processing like that typically means that you can support reverting back to the previous version, before the last phasing-in part.  The hard switch over makes it less likely that you can simply “turn the old system back on.” 

Analyzing the Analysis

A rough understanding of the two approaches, and the costs and benefits of each approach, will generally lead to a clear “winner” where one approach is demonstrably superior to another.  In those other cases, you should give the advantage to that approach that demonstrates a closer adherence to modern architectural design – which would most likely be the replacement application.  This becomes the decision maker because of its impact to scalability, reliability, security, and cost, all of which define a modern cloud-based architecture.  This circles back to my earlier point on how the return on investment for a refactor must be higher than the return on investment for a rewrite\replace, as there is always a larger error when understanding the effort of a refactor as compared to green-field development.  You need to consider this fact when you perform your evaluation of the ROI of refactoring your application.  Otherwise, you will mistakenly refactor when you should have replaced, and that will lead to another horror story.  Our job is already scary enough without adding to the horror…

 References

Cedrim, D., Gheyi, R., Fonseca, B., Garcia, A., Sousa, L., Ribeiro, M., Mongiovi, M., de Mello, R., Chanvez, A. Understanding the Impact of Refactoring on Smells: A Longitudinal Study of 23 Software Projects, Proceedings of 2017 11th Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering, Paderborn, Germany, September 4–8, 2017 (ESEC/FSE’17), 11 pages. https://doi.org/10.1145/3106237.3106259

Ferreira, I., Fernandes, E., Cedrim, D., Uchoa, A., Bibiano, A., Garcia, A., Correia, J., Santos, F., Nunes, G., Barbosa, C., Fonseca, B., de Mello, R., Poster: The Buggy side of code refactoring: Understanding the relationship between refactoring and bugs.  40th International Conference of Software Engineering (ICSE), Posters Track. Gotherburg, Sweden. https://doi.org/10.1145/3183440.3195030

Single Page Applications and Content Management Systems

This weekend, over beers and hockey, I heard a comment about how many Content Management Systems (CMS) do not support the use of modern single-page application (SPA) approach.  Normally, this would not have bothered me, but since this was not the first time I heard that point this holiday season, I feel like I need to set the record straight.  A CMS manages content.  The CMS provides a way to manage that content, include placing it on a page.  All of that work is done within the constraints of the CMS tool (examples include Sitecore, Joomla, Adobe Experience Manager, Drupal, WordPress, etc.).  What that CMS does with that content, however, is completely up to the developer/implementer.

A well-designed CMS system depends upon a collection of components.  These components each fulfill a different set of needs, either layout\design (such as a block of text with a title and sub-title as one component and an image, header, and block of content as another component) or feature\functionality (email address collection form, credit card payment section).  Each of these components support different content.  This means that a component that is a title, a sub-title, and a block of content must be able to support different text in each of those areas.  Creating a website is as simple as selecting a page template (there may be more than one) and then using the CMS functionality to put the various components onto the page.  An example of what this could look like is shown below:

An example page broken down into various content sections

The de-facto way to manage this is for the developer to write one or more HTML\CSS page-level templates, which act as the container for the components that site authors put into onto that page.  Each of those components is defined the same way, as snippets of HTML\CSS.  This means that the request\response cycle looks like:

  1. User clicks a link into the website
  2. Request made to URL
  3. CMS determines page for request, extracts definition
  4. Pulls out page template HTML
  5. Inserts all component HTML as defined, adding any author-controlled data such as the text for the title, which image to display, etc.
  6. Responds with completed HTML
  7. User presented with completely new page

That this is the general approach is what leads to the assumption that CMS systems do not support SPA-type designs.  This assumption is wrong. Virtually all enterprise-level CMS systems will support a non-page refresh approach to design, and all it takes is thinking outside the “de-facto” box.

One example of thinking about this process differently:

Consider an approach where a link on the menu does not request a new page, but instead makes a RESTful request to the CMS that returns information about what components belong on the “selected page” and where they need to be placed on that page.  Assuming each component is able to populate itself (perhaps through an AJAX approach after the component gets the ID that defines its content and was provided in the original RESTful request in response to the click on menu item), then it becomes a simple programming problem to assemble it into a coherent structure that is completely CMS managed.  The flow below walks through what this could look like.

  1. User clicks a link into the website
  2. RESTful request made to URL
  3. CMS determines page for request, extracts definition
  4. Provides page template identifier
  5. Adds information about the components
  6. Responds by sending a JSON object that describes the page layout
  7. Client-side processes JSON object and includes all necessary components (HTML & business logic)
  8. Rendering starts for user
  9. Each component makes an AJAX query to get its own content (if necessary)
  10. User presented with fully-rendered page without a complete page refresh

Using this kind of approach means that the JSON object returned in (6) to represent the pay shown earlier may look something like this:

{
"pageTemplate": standard,
"content": [{
"component": {
"template": "title_subtitle",
"contentId": 12875342567,
"sortOrder": 1
},
"component (copy)": {
"template": "email_signup",
"contentId": 12875618367,
"sortOrder": 2
},
"component (copy 2)": {
"template": "calendar_events",
"contentId": "129UY54A7",
"sortOrder": 3
}
}]
}

Which should be simple to process, even in that un-typed crap show that is Javascript.

I know what you are thinking, “Hey Bill, if it’s so simple, why isn’t everyone doing it”.  That is actually easy to answer.  There are many reasons why a company would choose NOT to build a SPA-like façade over their CMS.  In no particular order:

  • Complexity – It is easier and cheaper to write HTML-based components where the CMS interlaces them for you.  That is what CMSs do – they manage the relationship of content.  Abstracting that out to a client system adds complexity.
  • Editor support – Many CMS systems use WYSIWYG editors, so there is a need to have a defined HTML structure in a component so authors can see everything as they build it.  Since that structure has to be there already, any changes to client-side rendering is all new work.
  • Requires simple graphical design – the more complex the graphical layout of a site, the more complex a SPA wrapper would need to be.  The example that was used above is simple, one column that contains several components.  What if there are multiple columns in a page?  What if one of the components is a multi-column control that allows the author to put multiple columns of content into a single page column?  As the components get more complex, so will the work you have to do on the client to process them correctly.
  • Search-engine optimization – The whole point of a CMS is to have fine control over messaging and content.  A SPA-like approach makes SEO optimization problematic as search robots still have problems understanding the interaction between page changes and content changes – so cannot properly index content.
  • Analytics – Most analytics tools will just work when looking at sites where new pages load in the browser.  They will not “just work” in a SPA environment, instead you will generally need to add client-side logic to ensure that clicks are properly understood, with the correct context, by the analytics package.
  • Initial load – the first visit to a SPA application is long, generally because all of the templates need to be downloading before the first page can render.  There is a lot of downloading that would need to happen in an enterprise-level website with 10 page templates and 100 different components.  Yes, it could be shortened by using background or just-in-time loading, but are you willing to take a slow-down in an ecommerce environment where the first few seconds of the interaction between visitor and site are critical?
  • 3rd party add-ins – Analytics was mentioned earlier as being a potential issue.  Similar problems will happen with common third party add-ins like Mousify or Optimizely; the expectations that those tools have around working within a well-understood DOM will affect the usefulness of those tools.

I know, I spent the first part of this post talking about how it is possible to do this, and then spend the second part seeming to say why it should not be done.  That is not what that second section is saying.  Other than the first three bullets, all of these points are issues that anyone considering a SPA site should understand and account for in their evaluation, regardless of a CMS.

What it really comes down to is if, after understanding the implicit limitations imposed by a SPA approach, you still feel that it is the best way to design your website, then rest assured that your CMS can support it.  You simply need to change the way that your CMS delivers content to match the new design paradigm. The easiest approach to understanding this is to build a simple POC.  There are two “web pages” below.  It is made up of 1 page template and 4 content components: block of text with title and sub-title, an email sign-up form, a calendar, and an image.  Note that the second page has 2 instances of the block of text component with different content. 

If you can make this work in your client-side framework d’jour, than you can do it with your CMS.