Saturday, November 16, 2019

DSL high level concepts for orchestration - the flow graph is multi-leveled football

The flow graph is multi-leveled football (without defense)

High level concepts:

fan-out
fan-in
wait-for-signal
wait-for-worker

We need to make a graphical DSL that expresses this language.  But first, get the textual version on paper.

Delegation wrapper would keep the interface consistent from component to component (image/method/version) is coordinate for a method call.

This allows multiple method versions in the system at once.

Quarterback component would be written in Go and support orchestration of method calls
to image/method/version mappings.  Maybe in Gloo?

Target containers could be written in any language and support the (image) method version semantics.  Physical description of
image/method/version

image1/version1/method1
image1/version1/method2
image1/version2/method1

image2/version1/method1
image2/version1/method1
image2/version1/method2

The container is a combination of image and version.

Must implement GRPC contract to live in the system.  Have an interface that will accept method calls by name with Generic Input.

(Somebody has to have an example of this somewhere)

What is the equivalent of object (anything) in GRPC ?

This would allow the orchestrator component (quarterback) to make a set of nodes in a tree-like structure (implemented with channels and GoRoutines)

waitForSignal - wait for worker would represent goRoutines that wait for signal to run itself (carries the ball)

waitForWorker - would accept a signal that a worker was ready and deliver data on being triggered (catches the pass)

fanOut - replicate the ball and hand to several runners at once

fanIn - reverse hand off (or pass) accept ball back from the runner (flee flicker) then hand of to future component

You orchestrate plays linking these 4 types of connectors.

A "Play" has the ultimate objective where the quarterback throws pass outside the container to finish the orchestration.

The quarterback may as well subscribe to /image/version/method call semantics also as a system could benefit by multiple possible paths QBs

Its like football on several levels at once.  (similar to 3D chess in Star Trek)


Now we have two different implementations for the "flow": in the first case we are orchestrating calls to (service/deployment/pod), and in the other, we are orchestrating containers in a single pod.

It would be interesting to see how much benefit we could get from an abstract portion in the DSL.  Being able to describe the orchestration itself in a generic method would decrease the learning curve and make the system more intuitive.



No comments: