ExoSense Insights Schema¶

Last Update: 2024-11-19

Overview¶

This schema defines the interface for Exosite's streaming 'Insight Interface' which provides the ability to build logical operations and analytics in the form of transform, rule, and action functions that can be used in the ExoSense data pipeline to handle digital asset signal data.

This information specifies the required functions and data scheme for building an External Insight. External refers to a separate service that supports the required endpoints as defined below. This schema information may also be used to understand the data flow and objects for the ExoSense Inline Insights.

Insight modules contain one or many functions (one or multiple types). Each function provides it's own information - in this way the interface is dynamic for loading into the ExoSense user interface.

A new Insight module is defined by a swagger file and published as a ExoSense Insight service in Exosite's IoT Marketplace.

Download an example swagger template file

Reference & Definitions¶

API Paths¶

There are three required paths that an Insight must support:

• GET /info: Insight Module Info
• POST /insights: Function List
• POST /process: Processing Signal Data

Optionally, an Insight can support the following paths:

• GET /insight/{function_id}: Specific Function
• POST /lifecycle: Lifecycle Events

Insight Module Info¶

The GET /info endpoint serves to retrieve information about an Insight and is expected to return the following payload keys:

Example:

{
  "name": "Acme Co. Insight",
  "group_id_required": true,
  "description": "Insight exposing algorithms to predict the life of your Acme product.",
  "wants_lifecycle_events": true
}

Function List¶

The POST /insights endpoint returns a list of applicable Functions exposed by the Insight and information about those Functions.

The request body can have the following keys:

Example:

{
  "group_id": "CH00123",
  "limit": 4,
  "offset": 0
}

The response body should have the following keys:

See Insight Function Info

Example:

{
  "total": 7,
  "count": 4,
  "insights": [
    { "name": "Function1", ... },
    { "name": "Function2", ... },
    { "name": "Function3", ... },
    { "name": "Function4", ... }
  ]
}

Processing Signal Data¶

The POST /process endpoint will be called with the Signal Data as specified in the section above. What the Insight Function does with that payload is up to its author, but it is expected that this endpoint will return an array of arrays.

The inner array is for output Signal Datapoints for a specific Outlet. In the future, the outer array will be used to hold different Outlets' inner arrays.

A Function that returns nothing will respond with an empty array of arrays: [[]].

Specific Function¶

The GET /insight/{function_id} endpoint returns a single Function Info block corresponding to its member Function with the same function_id.

Lifecycle Events¶

The POST /lifecycle endpoint is optional and serves to inform the Insight about creation and deletion events.

This endpoint will receive a body with the following keys:

Core Payload Objects¶

The two primary payloads encountered with Insights, the Insight Function Info and Signal Data objects, are described here in detail.

Insight Function Info¶

Insight Function Info is used to generate the ExoSense UI when adding a Function to a Signal and to inform the ExoSense Pipeline on what information the Function needs to calculate a result.

The object has the following keys:

Type¶

Insight Functions can be one of three types: transform, rule, or action. Their classification into one of these buckets determines how ExoSense and the Pipeline treats them. The default type is transform if not specified.

Inlets¶

The inlets key is an array of Inlet objects, each of which can have the following keys:

Outlets¶

The outlets key is an array of Outlet objects, which can have the following keys:

Constants¶

Constants inform the ExoSense UI about parameters that users will supply when they add the Function to one or more Signals. Each Constant in the constants array can have the following properties:

Example:

{
  "constants": [
    {
      "name": "days",
      "type": "number",
      "description": "The number of previous days to analyze",
      "default": 1,
      "maximum": 7,
      "minimum": 1
    },
    {
      "name": "aggregation_function",
      "type": "string",
      "description": "The aggregation function to use",
      "enum": ["avg", "max", "min", "count", "sum"],
      "default": "avg"
    }
  ]
}

Signal Data¶

When an Insight Function receives Signal Data, the object the Function receives has three to five top-level keys:

ID¶

The ID is the specific instance of the Insight Function. If an Asset has four signals—A, B, C, and D—and they are Inlets to a Transform Function like the following:

A --\
     |-- Transform -- AB
B --/

C --\
     |-- Transform -- CD
D --/

Then the first and second Transforms will have different IDs. In the ExoSense Pipeline, these instances are known as Linkages, and the ID is the Linkage ID.

The ID field is critical for multiple-Inlet Insight Functions, as that Function will receive the Inlet Signal Datapoints separately and will need to keep track of their relationships.

Data¶

Signal Data passed around ExoSense to and from an Insight has a common schema.

Signal Data objects have the following keys:

Tags¶

Tags include general info about the Signal Datapoint as well as inlet, which ties an individual Datapoint back to its Inlet using the Inlet tag.

{
  ...
  "tags": {
    "resource": "data_in",
    "pid": $product_id,
    "metric": $data_in_key,
    "identity": $device_identity,
    "data_unit": $data_unit,
    "data_type": $data_type,
    "primitive_type": $primitive_type,
    "inlet": $inlet_tag
  }
}

Origin

Origin describes the source of the Datapoint. Frequently this source is a Device Channel, but could also be from a Signal injection, in which case the Origin will be the UUID of the Signal.

The most common origin is in the following format:

<PRODUCT_ID>.<DEVICE_ID>.<RESOURCE_ID>.<CHANNEL_ID>

For most situations in ExoSense, the <RESOURCE_ID> is data_in.

Args¶

The Args block in Signal Data contains the following keys:

Function Type Information¶

Transforms¶

Transforms are used to convert, evaluate, and process one or more streaming signals for purposes of data conversion or analytics. The outcome is one or more signals that can be used for visualization, further analytics, rules, and reporting.

Rules¶

The use case for rule type functions is to evaluate signal data and determine if a condition status has changed, which the most basic examples are for a value to be equal or not equal to a constant value or if a numeric value has crossed a threshold.

{
    "name": "SomeRule",
    "description": "SomeRule description",
    "type": "rule",
    "constants": [
      {
        "name": "const1",
        "description": "desc for const1",
        "type": "string"
      },
      {
        "name": "level",
        "description": "The alert level if the substring is found",
        "type": "number",
        "enum": [1, 2, 3, 4],
        "default": 1
      }
    ],
    "outlets": {
      "primitive_type": "JSON",
      "data_type": "STATUS",
    }
}

Status Data Type¶

The output of a rule insight is a JSON string.

Status Levels¶

User defined addendum messages¶

The notifications and rule event logs may include an extension message that can be added by an end user. To enable this per Rule function, the function must include the following reserved named constants.

  {
    name: "messageMatch",
    description: "Extra details for when value does match",
    type: "string",
  },
  {
    name: "messageElse",
    description: "Extra details for when value does not match",
    type: "string",
    default: "The value didn't match",
  },

Refer to Constants for details on properties.

Actions¶

The use case for action type functions is to send a rule status change value(a condition event) to a service that will consume and use that information to take some action such as send a notification, create a maintenance ticket, or register the condition for historical purposes.

{
    "name": "Some Action",
    "description": "Some Action description",
    "type": "action",
    "constants": [
      {
        "name": "const1",
        "description": "desc for const1",
        "type": "string"
      }
    ],
    "inlets": [
      {
        "primitive_type": "JSON",
        "data_type": "STATUS",
      }
    ]
}

Change log¶

• Deprecate and remove of Callback (cbi) functionality
• Add information for enum_presented and multiple key options for Constants.
• Deprecate and full removal of support for including signal history in the /process operation. Recommendation: Store history and/or state in the external insight service.
• Adding 'action' function type support
• Move content sections around with api routes first, adding overview, and table reformating.
• New document to detail insight transform and rule interface with ExoSense