Keeping up with Industry Advancements

2020 Think week turned out to be a starting point for more research.  As the bigger picture begins to emerge (see The Uber Language of Compute), there are many pieces of the puzzle that need further investigation:

Hexagonal Services in Go
GoKit
Envoy
GRPC
Go Scheduler
Gloo
Best Practices in Go

I created another playlist for going deeper into these areas.

Also, there appears to be some great info on Envoy in this playlist from EnvoyConf 2019

There are some interesting projects for simulating and testing Kubernetes Scheduler:

https://github.com/pfnet-research/k8s-cluster-simulator
https://github.com/kubernetes-sigs/cluster-capacity
https://github.com/volcano-sh/volcano/blob/master/docs/development/prepare-for-development.md

Update 1/19/2020 - A very interesting video from the developers at Hauwei could shed some light on to optimizing the mesh of computing resources in response to workload requirements.  As stated above, during some invocations, it may be possible to use cached values instead of recomputing partial answers.  This would imply that these runs require fewer resources than in the case where all the calculations need to be performed.  As it turns out, Hauwei developers have found a way to increase/decrease pod resources at runtime.   

https://youtu.be/eic6kBa3QAc

It also shows how to "restart" a pod - effectively flushing out any memory used by the pod - this could apply to new snapshots as previously it was envisioned that a pod should only be exposed to data from one snapshot.  This restart ability could provide the mechanism to roll over a set of resources to a new snapshot.

Another addition:  this paper shows a way to customize the Kubernetes scheduler using a Scheduler Extender:

https://drive.google.com/open?id=1qxgzzOaYYXqIrtrXSQSjnYPlKCRU3DCw

Implementing a “scheduler extender” process that the default KS calls out as a final step when making scheduling decisions

The Volcano.sh docs https://volcano.sh/docs/ cover a similar area.