Iterative refactoring and the emergence of Higher Order Patterns

 Higher Order patterns emerge during an iterative approach to refactoring.  Higher Order patterns are patterns on top of patterns.

The process I go through is basically Kent Becks canonical process with a mix of Neal Fords methodology :  1: Make it work. (raw patterns)  2: Make it work right (harvested patterns)  3: Make it work fast (optimized patterns)

Going through this process I have found that Higher Order patterns start to emerge.

I recently refactored a Python application at work.  This was the first time I had gone through a pattern harvesting process in Python and found that its essentially the same process as in Java.  In previous Java work, I had focused on code generation.  In this Python work, I implemented classes by hand.

As a first phase in refactoring, I look at the current code and develop a conceptual model.  (https://en.m.wikipedia.org/wiki/Conceptual_model).  It is this conceptual model that I use as a blueprint for the working code.

Interesting to note, the two modeling systems involved different domain.  The java system acted as a meta model for the generation of a family of applications involving reporting.  The python system involved creating test specifications for execution by a processing engine.

In both scenarios, data flows were identified and template processes were created to process run-time instances of domain model concepts.  Template methods define a standard processing flow that can be reused by many process variants.  This was the only common point between systems.


What is interesting, is that in coding solutions conforming to the derived conceptual model new patterns emerge in the form of "usage" patterns.  For instance, you may discover that 2 components have a closer relationship than was thought previously.  Or, you may find that a specific ordering in execution of components is optimal.   It is these patterns that become the target of the next refactoring step.  I refer to these patterns as "Higher Order" patterns; these patterns revolve around the usage of the first set of patterns abstracted from your raw program.

Sometimes these new patterns can be expressed in the modeling tool with meta-model validation (if code generation is part of the process) or enforced by using interfaces or by proper use of the type system.

The refactoring process helps you first understand your raw process and then iteratively, you start to understand your abstractions. The result is Higher Order patterns.