Getting the correct data to the correct compute resource; that is the goal. Time sliced queries into something like MongoDB or MemSQL could be the path to precisely identifying computationally consistent islands of data. These islands represent the relevant state at any point in the lifetime of the data stream. Computationally consistent would mean we are taking into account larger contexts relating to sets of related calculations.
MemSQL is a great choice for a node level-local cache of data. Queries against this database could feed computationally consistent time slices on the same node quickly.
Many compute islands (Pods) can be fed by a MemSQL database positioned at the node.
As mentioned, it should be possible to cache data at the pod level in RAM. This would be even closer to the compute than the local database. Kinda like current memory - or scratch space for that generation of containers.
Ok so time for a check:
Being able to define compute structures to support time sliced compute environments could be delivered in Kubernetes via many APIs that expose client-go interfaces. Check.
Being able to accurately address the whole set of resources is delivered via Gloo apis (Envoy) Check.
Being able to push accurate data to the sliced compute environment in a timely manner is delivered via MemSQL. Check
Lightning in a bottle. Check (delivery is an afterthought)
No comments:
Post a Comment