Saturday, January 11, 2020

The Uber Language of Compute

Thought can be expressed as the realization of a flow of operations.  Each component in that flow being arbitrarily complex.  The overall language is an aggregation of 4 languages.  Execution, Data, Provisioning and finally Routing.

Pluggable Universe: {

      Execution:  Julia ( or some other calculation engine )
      Data:  MemSQL with wrapper interface
      Provisioning:  Go and K8's APIs and Volcano.sh

}

A Universe is defined by the implementation of its "laws of physics": Execution, Data, and Provisioning.

The Multiverse {

       (List) Routing: Envoy and Gloo
               routes:  (List) Pluggable Universe

}

The Multiverse is a system of routing to many sets of Pluggable Universes each solving a portion of a complex problem.   Each set of Pluggable Universes is a set of mirrored time-sliced replicas that satisfy the laws of the Pluggable Universe definition but are shifted in time.

After optimizations are done at the level of logic flow, we must turn our sights to delivering streams of current results.  In this world data is short-lived, a particular data set has a shelf life.  The decision to provide a service that minimizes the window of outdated data is now possible.   As it turns out, the realization of windowed APIs is now possible,  and the amount of time spent in the window of outdated data is now configurable; the density of the computing power in a given context can vary as the number of dimensions in the multiverse is variable.




Picture each row of universes here as being a set of time-sliced versions.

Each new row is another component in the multiverse.  A set of Pluggable Universes all with the same goal.

Note: if you look at columns in this picture - left to right - they are all synced to the same time slice - so they are consistent with respect to source data.






No comments: