Skip to content

External Insights

An external insight module is a stand-alone software service that runs as a webservice, supporting the specified HTTP endpoints as defined by the Exosite Insight Schema.

Insight Integration Schema

To summarize, this webservice implements functions for:

  • Getting high-level information about the Insight
  • Getting a list of available functions, types, and input/output parameters which allows ExoSense to dynamically load the user interface
  • Getting info about a specific function, along with its required parameters
  • Process incoming Signal data (i.e. "run the function")

Hosting

The custom Insight services can be hosted anywhere that public HTTPS endpoints are available, including:

  • Exosite's Murano Application Platform (A template is provided)
  • Amazon Web Services
  • Microsoft Azure
  • Google Cloud Products
  • Heroku

Please contact Exosite support for more information on this advanced topic. Templates are available in Marketplace and through support.

The code behind the functions in Insight service is entirely up to its creator as long as it conforms to the Swagger definition and Insight Schema. The code that makes up an Insight can be written in any language available within the chosen host.

User Experience

An end user who has the Asset Management permission can use these functions in their pipeline configuration for the asset. They will be provided with a list of modules and functions to choose from including the custom functions. Users are then able to choose matching signals and define any constants.

Example showing a custom function configuration

Getting Started

The typical steps to create an external insight are as follows:

  1. Decide on where your function code will run and HTTPS webservice endpoints will be served.
  2. Implement the webservice required functions, per the Insight Schema.
  3. Publish a new ExoSense Insight Service in Exosite's Marketplace for your business account, using the Swagger template found below.
  4. Add your Insight service from the IoT Marketplace to your ExoSense instance to test.
  5. Add and update your functions.
  6. Note that the Insight service in the marketplace does not need updating unless the hosting location has changed. ExoSense pulls in the list of functions, function types, and information about the functions dynamically.

Using Murano to Host your Insight Service

You can build and host your custom Insight Module service as a Murano application. Note: You must have an advanced platform business account to build custom Murano applications.

The following template can be deployed from the Exosite IoT Marketplace. Lua-based example built to run in Murano

Look for ExoSense® Insight Module Template under the Murano Applications category.

Contact support for templates to run in other infrastructure.

Publishing

An Insight must be published in Marketplace (private or public) before it can be used within ExoSense

An External Insight service is published as a specific type of service (ExoSense Insight Service) in the Exosite IoT Marketplace using Murano's OpenAPI Swagger support.

Hint

When using the Murano Insight Module Template, the swagger YAML definition is automatically published to the endpoint /interface. For example https://mycompanyuniqueinsights.apps.exosite.io/interface

If you host outside of Murano, you need to have your swagger yaml file available for publishing the Insight Service.

Learn more about Publishing to Exchange IoT Marketplace

ExoSense Insight Swagger Notes

  • Very little needs to be changed from the template swagger, as the function end points are already defined.

  • It must use the element tag in the swagger definition as shown below for ExoSense to identify it as a service it can consume

    tags:
  - name: insight
    description: Insight Module

  • To add additional Solution level configuration parameters such as API keys and tokens to be used by the service, use the x-exosite-config-parameters key in the swagger file, info section.

    x-exosite-config-parameters: # optional exosite service configuration parameters
  - name:         auth_token
    description:  Describe this exosite service level configuration parameter, like auth token
    type:         string
    format:       password
    required:     false
Example Insight Swagger Template

Download this example swagger template file

swagger: "2.0"

info:
  version: "1.0"
  title: My Nifty Insight
  description: |
    This describes an Insight Module that can be added to Murano Exchange.  An Insight Module can
    implement one or more Insight Functions.  These Insight Functions can then be added to Assets
    as transform, rule, and action functions.
  contact:
    name: Your Name Here
    email: YOU@BIZ.com
  x-exosite-config-parameters: # optional exosite service configuration parameters
    - name:         auth_token
      description:  Describe this exosite service level configuration parameter, like auth token
      type:         string
      format:       password
      required:     false
host: a.real.host # Set this to the host your function is on
basePath: / # Set this or the path according to your function
################################################################################
# Do not edit below
################################################################################
tags:
  - name: insight
    description: Insight Module
schemes:
  - https  # Only https is supported.
consumes:
  - application/json # Only JSON is supported.
produces:
  - application/json # Only JSON is supported.

################################################################################
#                                Operations                                    #
################################################################################
paths:
  /info:
    get:
      operationId: info
      description: Get some info about this Insight
      responses:
        "200":
          description: Insight Functions successfully listed
          schema:
            $ref: "#/definitions/InsightInfoResults"
        default:
          description: Errors
          schema:
            $ref: '#/definitions/ErrorResponse'

  /insights:
    post:
      operationId: listInsights
      description: Get a list of available Insight Functions and info about them
      parameters:
      - name: body
        in: body
        description: Get a list of available insight functions
        required: true
        schema:
          $ref: '#/definitions/InsightsFilterParams'
      responses:
        "200":
          description: Insight Functions successfully listed
          schema:
            $ref: "#/definitions/InsightListResults"
        default:
          description: Errors
          schema:
            $ref: '#/definitions/ErrorResponse'

  /insight/{function_id}:
    get:
      operationId: infoInsight
      description: Get info about one Insight Function
      parameters:
        - name: function_id
          type: string
          in: path
          description: Identifier of function
      responses:
        "200":
          description: Insight Functions successfully listed
          schema:
            $ref: "#/definitions/InsightInfo"
        default:
          description: Errors
          schema:
            $ref: '#/definitions/ErrorResponse'

  /lifecycle:
    post:
      operationId: lifecycle
      description: |
        Notifications of when a linkage that will call the process function is created or deleted.
      parameters:
        - name: body
          in: body
          description: Lifecycle event
          required: true
          schema:
            $ref: '#/definitions/LifecycleEvent'
      responses:
        default:
          description: The response to this is ignored.

  /process: # The actual path can be anything.
    post:
      operationId: process
      description: |
        Your function to process a bunch of Signal Data.

        It is good form to copy all the fields of a source SignalData into the new SignalData,
        then update the properties according to your function.
      parameters:
        - name: body
          in: body
          description: Data to process and arguments on how to process it
          required: true
          schema:
            $ref: "#/definitions/SignalDataObjectArray"
      responses:
        "200":
          description: Data successfully
          schema:
            $ref: "#/definitions/SignalDataArrayArray"
        default:
          description: Error that stops the pipeline.
            Non-fatal errors should produce new signals on a separate outlet.
          schema:
            $ref: '#/definitions/ErrorResponse'

################################################################################
#                                 Definitions                                  #
################################################################################
definitions:
  InsightInfoResults:
    type: object
    description: Info about this Insight Module
    properties:
      group_id_required:
        type: boolean
        description: True if the listInsights requires group_id to be something to list functions.
      name:
        type: string
        description: Presented name
      description:
        type: string
        description: Additional text about this insight. Markdown can be used.
      wants_lifecycle_events:
        type: boolean
        description: |
          True if this insight wants to be notified when linkages using it are created and deleted.
          Defaults to false.
      translations:
        type: object
        description: Translations of the name and description fields
        additionalProperties:
          type: object
          description: Keyed by each translated language code 
          properties:
            name:
              type: string
              description: Presented name
            description:
              type: string
              description: Additional text about this insight. Markdown can be used.
    required: [group_id_required]

  InsightsFilterParams:
    type: object
    description: Filter options for insight functions
    properties:
      group_id:
        type: string
        description: |
          Which group of insight functions to get.  The meaning of the contents is up to the Insight Module.
          Including ignoring it and always returning all.
      limit:
        type: integer
        description: Limit how many insights to fetch
      offset:
        type: integer
        description: Offset into the returned insights
    required: [group_id]

  SignalTypeInputInfo:
    type: object
    description: |
      What data type the incoming Signal must have.  One of `primitive_type`, `data_type` or `data_unit`
      should be specified.
    required: [name]
    properties:
      primitive_type:
        type: string
        enum: [NUMERIC, STRING, BOOLEAN, JSON]
        description: If present, the primitive type that the connecting signal must have
      data_type:
        type: array
        description: If present, the type that the connecting signal must have
        items:
          type: string
          description: A required data type
      data_unit:
        type: array
        description: If present, the units that the connecting signal must have
        items:
          type: string
          description: A required unit.
      tag:
        type: string
        description: Optional tag to add to the SignalData for inlet matching
      name:
        type: string
        description: Name of this signal
      description:
        type: string
        description: Optional text to help users.
      translations:
        type: object
        description: Translations of the name and description fields
        additionalProperties:
          type: object
          description: Keyed by each translated language code 
          properties:
            name:
              type: string
              description: Presented name
            description:
              type: string
              description: Optional text to help users.

  SignalTypeOutputInfo:
    type: object
    description: |
      What data type the incoming Signal must have.  One of `primitive_type`, `data_type` or `data_unit`
      should be specified.
    properties:
      primitive_type:
        type: string
        enum: [NUMERIC, STRING, BOOLEAN, JSON]
        description: If present, the primitive type that the connecting signal must have
      data_type:
        type: array
        description: If present, the type that the connecting signal must have
        items:
          type: string
          description: A required data type.
      data_unit:
        type: array
        description: If present, the unit that the connecting signal must have
        items:
          type: string
          description: A required unit.
      name:
        type: string
        description: Name of this outlet
      description:
        type: string
        description: Optional text to help users.
      suggested_name:
        type: string
        description: |
          A suggestion for what to name signals that get created for this outlet.
          This may include `{}` to indicate where to place the function name.
      translations:
        type: object
        description: Translations of the name and description fields
        additionalProperties:
          type: object
          description: Keyed by each translated language code 
          properties:
            name:
              type: string
              description: Presented name
            description:
              type: string
              description: Optional text to help users.
            suggested_name:
              type: string
              description: |
                A suggestion for what to name signals that get created for this outlet.
                This may include `{}` to indicate where to place the function name.

  ConstantInfo:
    type: object
    description: A constant that needs to be provided by the user
    properties:
      name:
        type: string
        description: The name of this constant; This is not shown to users and is the code identifier for this constant.
      type:
        type: string
        enum: [string, number, boolean]
        description: What kind of value this constant accepts
      description:
        type: string
        description: Some words for the user about this constant
      default:
        type: [string, number, boolean]
        description: Optional value to use if user doesn't specify one.
      enum:
        type: array
        description: An array of allowed values.
      enum_presented:
        type: array
        description: If present, the strings to display instead of the values in `enum`.  The length of this must match `enum`.
        items:
          type: string
          description: Presentable name for one value
      maximum:
        type: number
        description: For number type, the largest it can be
      minimum:
        type: number
        description: For number type, the smallest it can be
      multiple:
        type: boolean
        description: |
          When this is true, this constant will be an array of values instead of a single value.
          The UI will present the user a way to enter this constant multiple times.
      translations:
        type: object
        description: Translations of the name and description fields
        additionalProperties:
          type: object
          description: Keyed by each translated language code 
          properties:
            description:
              type: string
              description: Some words for the user about this constant
            enum_presented:
              type: array
              description: If present, the strings to display instead of the values in `enum`.
              items:
                type: string
    required: [name, type]

  InsightInfo:
    type: object
    description: One insight function
    properties:
      id:
        type: string
        description: The unique id for this insight function
      name:
        type: string
        description: A human friendly name for this insight function
      description:
        type: string
        description: A description of this insight function to display to users.
      translations:
        type: object
        description: Translations of the name and description fields
        additionalProperties:
          type: object
          description: Keyed by each translated language code 
          properties:
            name:
              type: string
              description: A human friendly name for this insight function
            description:
              type: string
              description: A description of this insight function to display to users.
      type:
        type: string
        enum: [transform, rule, action]
        description: |
          What kind of insight this function is.  Determines how it gets used and how the UI
          presents it.  If omitted, then 'transform' is used.
      constants:
        type: array
        description: Additional user specified values to pass to this insight function
        items:
          $ref: '#/definitions/ConstantInfo'
      constants_multiple_maximum:
        type: number
        description: |
          If there are constants with multiple set to true, this states the maximum number of times the
          constants can be repeated. (The maximum size of the arrays.)
          If unspecified then the UI can apply any limit, including none.
      inlets:
        type: array
        description: The kinds of data that can be connected. (This is an unordered set)
        maxItems: 5
        items:
          $ref: '#/definitions/SignalTypeInputInfo'
      outlets:
        type: array
        description: What kind of data each output will produce. (This is an ordered array)
        maxItems: 5
        items:
          $ref: '#/definitions/SignalTypeOutputInfo'
    required:
      - id
      - name
      - description

  InsightListResults:
    type: object
    description: The found insight functions with count info
    properties:
      total:
        type: integer
        description: The total number of insight functions in this group
      count:
        type: integer
        description: Number of insight functions returned
      insights:
        type: array
        description: The insight functions
        items:
          $ref: '#/definitions/InsightInfo'
    required:
      - insights
      - total
      - count

  LifecycleEvent:
    type: object
    description: A Lifecycle event
    properties:
      event:
        type: string
        description: What action caused this event
        pattern: "(create|delete)"
      id:
        type: string
        description: The unique id for the linkage that has the event.  This id will match in the calls to process.
      args:
        type: object
        description: The arguments for this linkage that will be also passed to each call of process.

  SignalData:
    type: object
    description: A single instance of a piece of data inside a signal.
    properties:
      tags:
        type: object
        description: |
          Tag-Value pairs to help ID the data. Used to tie the data back to an
          Asset or Device or other things.
        additionalProperties:
          type: string
          description: Value for a tag
          minLength: 1
      ts:
        type: integer
        description: Unix timestamp in microseconds of when the data originated
      gts:
        type: integer
        description: Unix timestamp in microseconds of when this SignalData was generated
      value:
        type: [string, number, object]
        description: The value for this instance of data
      origin:
        type: string
        description: The original Publishing ID
        minLength: 1
      generated:
        type: string
        description: The Publishing ID that created this SignalData.
        minLength: 1
      ttl:
        type: integer
        description: Value used to help prevent data from infinitely cycling
    required:
      - ts
      - value
      - origin
      - generated



  SignalDataArray:
    type: array
    description: Array of Signal Data
    items:
      $ref: '#/definitions/SignalData'

  SignalDataArrayArray:
    type: array
    description: Array of Array of Signal Data.  An array for each outlet, each outlet can have many signals.
    items:
      $ref: '#/definitions/SignalDataArray'

  SignalDataObjectArray:
    type: object
    description: The data and arguments to be processed.
    properties:
      id:
        type: string
        description: A unique ID for this instance of this function in an asset pipeline.
      data:
        $ref: '#/definitions/SignalDataArray'
      args:
        type: object
        description: Object of instance arguments.
        properties:
          function_id:
            type: string
            description: Which insight function to run
          group_id:
            type: string
            description: Which group_id this is under
          constants:
            type: object
            description: Additional constant parameters for the selected function


  ErrorResponse:
    type: object
    description: Error response that stops the pipeline.
    properties:
      message:
        type: string
        description: Error message
      status:
        type: number
        description: A numerical code for the error
      name:
        description: Error type
        type: string