Monday, December 23, 2019

Confirmation of Design Principles via Tekton

As it turns out, spending time to learn Tekton is providing useful information for the framework I have been talking about.  Coincidentally, Tekton's process of creating build pipelines is very similar to my proposed solutions for batch systems.

They have what they are referring to as a "hacky" solution to how to execute operations in order on the same pod.  I had proposed using GoFlow to create this execution graph and having one of the containers in the pod act as a "quarterback" making GRPC calls to the other containers in the graph.  Even though my idea doesn't match, it confirms that something needed to be created in this area. In addition to this, it appears that the overall idea of using cooperating containers within a Pod seems to confirm that I am heading in the right direction.

They have also identified the ability to use the file system to exchange data between containers in a Pod; I am taking this as validation of the concept.  As for exchanging data between pods, they are looking into having the scheduler schedule pods on the same node when performance is a key.  I am taking this as confirmation that I am heading in the right direction with respect to the Volcano.sh research.

All in all, it's encouraging to see that systems are being built using very similar designs.  In addition to researching the Volcano.sh codebase, it appears I have to start reading the Tekton code base.  




No comments: