Integration Testing, JUnit, Microservices, and Validation in Production Systems

In programming software, best practice states that developers should be able to run programmatic tests against code to determine the validity  of process results.  JUnit provides a good starting point for out of band testing (not during production flow) and some of the techniques used for testing in JUnit can be applied to in band (part of production flow) testing.

Standardizing Data Flow:

In developing JUnit test suites, it is often useful to be able to abstract repeating data flows inside JUnit to avoid code duplication; this particular problem is well suited to use Inner classes along with a template method to standardize data flow.

Breadcrumbs and Production Validation:

By persisting "breadcrumbs" (serialized sets of objects collected from in-band process results) it is possible to validate both the final result but also to validate the data at key points in the process flow.  As processes work to aggregate results into answers that are the commutation of many subsystems working together.  It is often the case when the final answer is wrong and the developer wants more information on why.  Via the analysis of "breadcrumbs" persisted along the way, total time to debug and fix issues an be decreased.

Production Validation via reverse Engineering:

Inline validation of production results can be done via "reverse" engineering expected results from expected input.  Pieces of a result can be compared against input values in an effort to avoid any obvious miscalculations.  This testing not replicating all the business logic for each process, but sometimes answers have certain characteristics that can be verified with (near in band) repeatable processes.

Microservices

In moving to micro-services, methods have to be developed to test multiple processes from a single JUnit based test.  Early research on this has pointed me in the direction of using ProcessBuilder to start defendant micro-services before individual unit tests are run (effectively bootstrapping the parts of the equation that are outside the current Spring application context).

Future Direction:

As I am building out my lab and plan to start practicing creating and testing data flows with MicroServices and Spring boot, I am trying to take an inventory of all the various ways to test and validate data in an effort to bake the maximum amount of testability into the designs i create.

Items for consideration:

• Bootstrapping micro-services
• Using spring boot heartbeat to verify status of micro-services prior to running tests
• Using Aspects to log and audit data
• aspects vs or together with breadcrumbs
• generic breadcrumb collection and rehydration
• collection of breadcrumbs without impacting performance (async possibly Kafka)
• Near Real Time Validation with Kafka?
• When to use templates and inner classes
• Are there flows that lends themselves to this kind of testing
• Is there anything that can be done in original design that would make this easier?