Mechanical Sympathy with K8's

Jackie Stewart said, “You don’t have to be an engineer to be be a racing driver, but you do have to have Mechanical Sympathy.”  He meant that understanding how a car works makes you a better driver.  (https://dzone.com/articles/mechanical-sympathy)

In a nutshell, understanding Kubernettes can make you a better programmer.   Your software can now communicate with the underlying infrastructure... new sets of containers can come and go based on application state.    The ability to drive the deployment of Kubernettes objects based on application logic opens up new doors.

It seems like there is another level of Mechanical Sympathy when you use K8's  You write code to run faster on certain hardware setups; this has been your job. Now, you can tell Kubernettes to lay out specific hardware to run workloads and have it provisioned on the fly.   This is not simple load balancing and horizontal scaling; that type of scaling expands in predetermined ways in a statically designed Kubernettes cluster.  What we are looking at now is the ability for apps and Kubernettes to work together more seamlessly. Yes, you still have scaling, but K8's now allows you to provision the optimal infrastructure needed for your workload and it can do it dynamically in response to application state.  Basically it boils down to this:  1. you don't have to layout your structure statically  2. your deployment topology can change as your applications needs change.

As per the Kubebuilder book. (https://book.kubebuilder.io/getting_started/why_kubernetes.html)

Building services as Kubernettes APIs provides many advantages to plain old REST, including:

• Hosted API endpoints, storage, and validation.
• Rich tooling and clis such as kubectl and kustomize.
• Support for Authn and granular Authz.
• Support for API evolution through API versioning and conversion.
• Facilitation of adaptive / self-healing APIs that continuously respond to changes in the system state without user intervention.
• Kubernetes as a hosting environment

Where could this lead us?  

Need an app that needs a db and a messaging layer... well...install the db and then maybe install zookeeper and then maybe get another framework to run our code... and an application server and on and on.  Without containerization these different pieces all live on different islands and there is no single abstraction that ties them together.

Along comes Kubernettes and now you can bring them together, and even better, the app itself can be aware of the compute mesh.  Oooooh...this is different

Now, an app can do things like spin up new data processing structures as they are needed... 

Who says an app can't direct the topology of the compute environment based on categorizing workload? 

As you get another specialized context to work on it's not sent to a generalized set of pre-existing services. Maybe it's sent to a sub-mesh ( group of K8's objects ) that may have a different blueprint - a sub-mesh ( group of k8's objects ) that didn't exist before the request, and a sub-mesh ( group of K8's objects ) that is optimized for a particular workload.

Addition (2_10_2019) - Envoy defines a subMesh as a "cluster"

Sub-meshes due to their dynamic nature, include the ability program data affinity. If you just defined this new mesh of compute resources to solve a problem then maybe you need some supporting structures to facilitate this work?  Maybe data can be loaded onto a single host, making it easy to get to; the app having super fast access to this information: mechanical sympathy.

If two sub meshes need to talk to exchange data, this is possible also:  layers upon layers of communication.  Even grouping sub-meshes to work together by placing data that needed between different parts of a problem - ON THE SAME MACHINE.

A few engineers at LinkedIn had published a paper talking about using Samza aggregate localized pools of data.  This led to faster processing as work was done in a local DB and then the result information was passed up back up the centralized DB.  See the picture below.

Note: this is only an example of a set compute constructs that are arranged to solve a particular problem: it is one possible blueprint that can be used to cookie cut solutions developers can create their own patterns.  I chose this example as it was designed with mechanical sympathy in mind.  Local databases are used as communication across tiers is expensive.  Also note this diagram does not depict kubernetes objects but instead depicts pieces of puzzle that could be containerized maybe using LiftBridge as the messaging layer: (https://github.com/liftbridge-io/liftbridge).  Also see: https://github.com/nats-io/nats-operator

Now think about this whole 2 DB setup in terms of kubernetes or a sub-mesh

http://www.vldb.org/pvldb/vol10/p1634-noghabi.pdf

Now we have the ability to locate components exactly where we want;  we can push data down to the individual machine where it needs to be and compute engines that need fast access to that data have it at their fingertips

The pic above can be one possible sub-mesh blueprint.  Now think about a grid of such sub meshes. Maybe bigger aggregation points? Taking in data from its sub meshes: layer on layer.  

Seems hard right?  Not if you can program the mesh creation now :)   I am not talking about entering kubectl commands to create static data structures or even using Helm to upgrade complex apps.

Maybe you can create something shaped like this (see below)  and have control over

:  what kind of hard disk is used on each component

:  how much cpu and memory are dedicated to each component

:  maybe you need to guarantee that each component has a GPU ?  - yeah we code that too

:  does your workload require a document DB ? maybe a key value store?  - we have you covered.

Note:  the grid below is homogeneous - but that does not have to be the case

Addition (2_10_2019) : Envoy does suggest that a cluster be closely related components.  

Maybe the future of application design is not in designing static kubernetes clusters?  Maybe its more dynamic than that?