Distributing The Load

One of the more interesting components of Bear is the use of Distributed Workflows.  In this post, we are going to go over some of the basic functions and show how these workflows can be used to reduce network traffic and increase operational responses.

In order to understand how workflows operate, it is important to note that every behavior in Bear is controlled through policy.  The core modules in Bear are very generic processing engines that enforce rules stemming from policies created upstream from the current device.  These policies enable rapid changes to new events such as data corruption, device malfunction and intrusions without any code revisions or firmware updates.

Within the general context of these policies is an array of possible response paradigms that tell a given device how to handle different events.  Simple commands include GET, PUSH, STORE and ALERT.  Layered on top of these simple commands are more complex options found in the Kodiak client which include PROCESS, BRANCH, LOGICAL IF, LOGICAL LOOP and other similar commands.  It is these latter commands, along with entries that define algorithms which enable simple localized workflows.  All of these commands have options that enable sophisticated routing and processing of data as needed.  Kodiak is also being built with a series of core math processing blocks that can be combined together to perform significant operations at each level of the system.  Thus a rollup can be performed throughout an enterprise and reduce the resultant data pushed to the cloud by magnitudes of order.

Data itself can be identified by a series of options from destination/source addresses to protocols and even core application.  While Bear is not yet providing API integration with applications, these operations can be used to reduce data flow with the main focus on systemic health.  The concept of system health will be covered in a blog later this week.  If an operation is sufficiently complex, the resources of a perimeter device might not be sufficient to complete the operation – this is where the distributed side of Bear comes into play.

Bear provides the ability to BORROW and PROVIDE resources that can be utilized by other devices depending on local configuration and resource usage.  Thus a given hub, with low processing loads, might be configured to PROVIDE a set percentage of its free processing time.  This discoverable message can then be consumed by, say, a sensor using a BORROW command and a complex component can be outsourced as needed.  Conceptually, such a process might look as follows:

COMBINE[Packet Set A + Packet Set B]:Result 1 => BORROW(ROLLUP[Result 1]),MIN[20]): Result 2

This conceptual rule states that two sets of packets should be combined, the logical for combining is left out for the sake of readability, and then rolled up (again the details are omitted).  The rollup can take significant resources – indeed at least 20% processor time needs to be available – so borrow resources from any other available device.  All of this commands run through the secure communication channels and thus everything is safe from an exploit.

This BORROW capability also enables nested workflows and can be used in the opposite direction such that a series of sensors might be asked to run a small workflow and submit a resultant set to a controller – all elicited by a controller to minimize overhead on the sensors.  This process also works upstream such that distributed workflows can be nested from the smallest sensor/effector endpoints through the cloud.

In this way, data can be processed efficiently on the fly, and automated responses such as reconfiguration of corrupt devices, data flow optimizations, and intrusion responses can occur at the extremes of a system while still interacting with higher-level enterprise systems as needed.  In this manner, Bear combined traditional and edge computing into one larger, distributed, workflow system.