CloudEvents Protocol¶

Introduction¶

The Exosite Cloud Federation API provides an authenticated interface for other device cloud platforms to federate device IoT data with Murano.

This document provides information about this federation API when used with the CloudEvents protocol, which uses CloudEvents HTTP Webhooks to interface with third party IoT clouds.

CloudEvents Specification

It is recommended that you familiarize yourself with the following referenced documents before proceeding as they provide context for CloudEvents.

Overview¶

Exosite's platform offers its Cloud Federation capabilities through IoT Connectors configured to accept, authenticate and handle connections from remote clouds. Remote clouds will be able to send data on behalf of the devices connected to it.

Device Identities: Cloud Federation vs direct Device API

Similar to directly connected devices, the IoT Connector will maintain state and status for such remote devices with a few exceptions:

Since the devices are not directly connected to the platform, connection status information may not be available for them, depending on whether the federated cloud forwards such information via the events below (optional).
Devices can auto-provision when the remote cloud sends data on behalf of them for the first time. In this case, the device identity does not need to exist in the IoT Connector in order for the remote cloud to be able to send data. This feature requires that the Auto Provisioning setting is enabled in the IoT Connector settings.

Setup¶

Typical use of this API protocol will follow these steps:

Exosite Platform: Create a new Cloud Federation IoT Connector (Use the CloudEvents template) and set the protocol to CloudEvents.
IoT Connector: Specify the Remote Cloud's Authorization token (paste into the connector protocol settings) and Origin
IoT Connector: Set any other settings regarding device identity provisioning (e.g. default auto create on data_in events)
Remote Cloud: Configure to send webhook CloudEvent requests to the IoT Connector's FQDN endpoint.
Remote Cloud: Send Webhook Validation request (optional - one time)

Normal Usage¶

After setup, the Remote Cloud is now able to send Webhook CloudEvents requests to the IoT Connector. The only required event is the exosite.identity.data_in event, as the IoT Connector can be set up to auto create device identities on data_in events. If this is not enabled, the remote cloud will need to send exosite.identity.created to create a device identity to be used by solutions and recommend also sending exosite.identity.connected events for solution use.

FQDN¶

The Cloud Federation API uses a single HTTP endpoint per IoT Connector, secured by TLS. The remote cloud will use this endpoint as a webhook to interface with the IoT Connector.

https://<iot_connector_id>.c2.exosite.io

Notice how it is a c2 domain (as opposed to device API IoT Connectors which are m2). Accordingly, these endpoints present a *.c2.exosite.io certificate.

Connection Handling

If the remote cloud keeps the connection to the IoT Connector open and sends multiple requests within the same connection, the requests will be distributed across the platform nodes. This may result in parallel processing, or even chronological inconsistencies, of events originating from the same remote device. The way this manifests is that a received value with a timestamp older than that in the device state (set and reported), will not overwrite the existing value. However, the value will still be sent to the event handler.

Authentication¶

Authentication of remote clouds connecting to the IoT Connector will happen at the domain level. Once authenticated, they are allowed to send data on behalf of their connected devices without said devices having to authenticate with the IoT Connector. That is because the platform inherently trusts that the devices will already have authenticated to the remote cloud.

The basic authorization is configured in the IoT Connector settings.

Currently only Bearer token is supported

Webhook Validation¶

Remote clouds may first send a Webhook validation request to the IoT Connector's FQDN endpoint to verify they are authorized to connect to the platform and understand the expected request rate.

The CloudEvents Webhook Validation is optional, the validation is not mandatory. The CloudEvents protocol defines it but it does not require that complying clouds call it before starting to make regular event requests. It provides the ability for the remote cloud to ensure that the cloud (Exosite Murano) it is talking to is who it says it is. It also offers a way to negotiate a call rate.

Validation request format

The CloudEvents Webhook Validation must be sent as a HTTP OPTIONS request.

curl -X "OPTIONS" "https://<unique_iotconnector_id>.c2.exosite.io/"
     -H 'WebHook-Request-Origin: <your_origin.example>'
     -H 'WebHook-Request-Rate: 120'
     -H 'Host: <unique_iotconnector_id>.c2.exosite.io'
     -H 'Authorization: Bearer H4sj8fbPFx86toDR'

For the role of WebHook-Request-Origin and WebHook-Request-Rate headers, refer to the CloudEvents HTTP Webhook specification. The Authorization header is used for authentication and currently only Bearer Tokens are supported.

Successful Response format

On successful validation, the IoT Connector will respond as such

HTTP/1.1 204 No Content
allow: POST
connection: close
date: Tue, 28 Jun 2022 13:15:30 GMT
server: Murano
webhook-allowed-origin: <your_origin.example>
webhook-allowed-rate: 100

This response from the IoT Connector is telling the remote cloud that it will accept CloudEvents from the specified origin at a rate indicated in the response which, in the example above, happens to be slower than requested (100 requests/minute vs. the requested 120). See the CloudEvents HTTP Webhook specification for further information about webhook-allowed-origin and webhook-allowed-rate.

Webhook Events¶

An authorized remote cloud can send CloudEvent events as POST requests to the IoT Connector's FQDN endpoint. There are different types of events supported (see below).

Only one remote cloud source is supported by the IoT Connector

The CloudEvents must be sent as HTTP POST requests.

CloudEvent event request format

curl -X "POST" "https://<unique_iotconnector_id>.c2.exosite.io/"
    -H 'Authorization: Bearer H4sj8fbPFx86toDR'
    -H 'Content-Type: application/cloudevents+json; charset=utf-8'
    -H 'WebHook-Request-Origin: <your_origin.example>'
    -H 'Host: <unique_iotconnector_id>.c2.exosite.io'
    -d '{
        "specversion": "1.0",
        "id": "014234de-0818-47c4-9bc4-1cb0bdf0302f",
        "source": "/yourplatform/application",
        "time": "2022-06-29T12:10:18+02:00",
        "type": "exosite.identity.data_in",
        "subject": "00001",
        "datacontenttype": "application/json",
        "dataschema": "#",
        "data": {
            "alias": "data_in",
            "timestamp": 1656702991,
            "value": "{\"channel1\":32, \"channel2\":56}"
        }
        }'

As seen in the above example, the request uses the FQDN endpoint, it uses Bearer Token authentication and the origin as set in the IoT Connector. The Content-Type header indicates that this is a CloudEvent packaged as a JSON object. The payload itself is specific to Exosite's implementation of the Federation API's CloudEvent protocol.

Response Format

The response to such an event delivery requests is seen below:

HTTP/1.1 204 No Content
connection: close
date: Wed, 29 Jun 2022 12:10:18 GMT
server: Murano

Event Types¶

In order to cover a device's full life cycle, the following event types are supported by Exosite's Federation API CloudEvent protocol:

exosite.identity.created
exosite.identity.deleted
exosite.identity.connected
exosite.identity.disconnected
exosite.identity.data_in

The naming of the events follows a reverse DNS-like scheme, as recommended in the CloudEvents Specification.

Common Fields

Some of the event fields are common across all event types. For further details, please refer to the CloudEvents Specification. Below is a brief summary of what each field represents.

specversion: always set to "1.0" as required by the current CloudEvents specification (enforced by the Exosite IoT Connector)
id: a random UUID v4 that is unique to each event produced by the remote cloud (currently unused)
source: the context in which the event originated on the remote cloud (currently unused)
time: the time when the event was generated in RFC3339 format (currently unused)

IMPORTANT: Although some of these fields are currently unused, their format is validated and are enforced to be in conformance to the CloudEvents specification.

Event Content Modes¶

The Federation API CloudEvents protocol supports all three event content modes: binary, structured and batched. For details, see CloudEvents HTTP Binding.

As per the CloudEvents HTTP Binding specification, batched content mode is just a JSON array of events in structured content mode. Hence, batched mode examples will not be given below. However, for batched content mode requests, don't forget to set

Content-Type: application/cloudevents-batch+json; charset=utf-8

Events¶

The following events are supported by the Exosite Cloud Federation API CloudEvents protocol.

exosite.identity.data_in¶

This event is sent when a remote device has sent data to the remote cloud.

data_in event vs data_in resource used by ExoSense

It's important to note that the IoT Connector has an event type named data_in which occurs when a new data event has occurred. When using the ExoSense condition monitoring solution, its schema requires that a resource also called data_in of type string is used for sending channel values.

Values sent to data_in and config_io resources for ExoSense use cases must use a JSON encoded string.

ExoSense Data I/O Schema

Structured content mode

curl -X "POST" "https://p1.c2.exosite.io/"
     -H 'Authorization: Bearer H4sj8fbPFx86toDR'
     -H 'Content-Type: application/cloudevents+json; charset=utf-8'
     -H 'WebHook-Request-Origin: exosite.cloud.test'
     -H 'Host: p1.c2.exosite.io'
     -d '{
          "specversion": "1.0",
          "id": "014234de-0818-47c4-9bc4-1cb0bdf0302f",
          "source": "/remote-cloud/application/devices",
          "time": "2022-06-29T12:10:18+02:00",
          "type": "exosite.identity.data_in",
          "subject": "00001",
          "datacontenttype": "application/json",
          "dataschema": "#",
          "data": {
            "alias": "data_in",
            "timestamp": 1656702991,
            "value": "{\"temperature\":43,\"pressure\":64,\"state\":\"on\"}"
          }
        }'

Binary content mode

curl -X "POST" "https://p1.c2.exosite.io/"
     -H 'Authorization: Bearer H4sj8fbPFx86toDR'
     -H 'Content-Type: application/json; charset=utf-8'
     -H 'WebHook-Request-Origin: exosite.cloud.test'
     -H 'Host: p1.c2.exosite.io'
     -H 'ce-specversion: 1.0'
     -H 'ce-id: 3770399b-f414-44c9-bc43-6176a5049abe'
     -H 'ce-source: /remote-cloud/application/devices'
     -H 'ce-time: 2022-06-29T12:06:44+00:00'
     -H 'ce-type: exosite.identity.data_in'
     -H 'ce-subject: 00001'
     -H 'ce-dataschema: "#"'
     -d '{
          "alias": "data_in",
          "timestamp": 1656702991,
          "value": "{\"temperature\":43,\"pressure\":64,\"state\":\"on\"}"
        }'

Notice how the Content-Type header is different between the structured and binary modes:

structured: it indicates that the payload is a CloudEvent in JSON presentation.
binary: it indicates the format of what would be the format of the data field in structured mode, i.e the payload.

Note: See the CloudEvents HTTP Binding specification for an explanation on why the difference.

Field definitions:

datacontenttype (structured only): The format of the data field content.
dataschema: This defines the internal structure of the data field (or that of the payload in binary mode)
data (or the payload in binary mode): The actual data as expected by the IoT Connector.
- The alias must exist as a resource on the IoT Connector (such as the ExoSense schema's data_in resource)
- The format of the value must match that configured for the resource.
- The timestamp is optional, a timestamp from the platform will be used if not provided.
subject: The identity of the device sending data.
type: exosite.identity.data_in

Actions taken by the IoT Connector:

If auto-provisioning is enabled and on the first connection ever:

An identity is created in the IoT Connector
The identity is placed in provisioned status
The provisioned event is generated in the platform

On all connections:

The identity state is updated according the payload, i.e. data_in or config_io resource is updated
The data_in event is generated in the platform

exosite.identity.created¶

This event is sent when a new device has been created on, or added to, the remote cloud.

Structured content mode

curl -X "POST" "https://p1.c2.exosite.io"
     -H 'Authorization: Bearer H4sj8fbPFx86toDR'
     -H 'Content-Type: application/cloudevents+json; charset=utf-8'
     -H 'WebHook-Request-Origin: exosite.cloud.test'
     -H 'Host: p1.c2.exosite.io'
     -d '{
          "specversion": "1.0",
          "id": "bf7e2043-0c3f-4727-aac3-cdad95d245e8",
          "source": "/remote-cloud/application/devices",
          "time": "2022-06-29T13:27:15+02:00",
          "type": "exosite.identity.created",
          "subject": "00001"
        }'

Binary content mode

curl -X "POST" "https://p1.c2.exosite.io"
     -H 'Authorization: Bearer H4sj8fbPFx86toDR'
     -H 'ce-specversion: 1.0'
     -H 'ce-id: 3770399b-f414-44c9-bc43-6176a5049abe'
     -H 'ce-source: /remote-cloud/application/devices'
     -H 'ce-time: 2022-06-29T12:06:44+00:00'
     -H 'ce-type: exosite.identity.created'
     -H 'ce-subject: 00001'
     -H 'WebHook-Request-Origin: exosite.cloud.test'
     -H 'Host: p1.c2.exosite.io'

Field definitions:

subject: The identity of the device having been created.
type: exosite.identity.created

Actions taken by the IoT Connector:

An identity is created in the IoT Connector
The identity is placed in provisioned status
The provisioned event is generated in the platform

exosite.identity.deleted¶

This event is sent when an existing device has been removed from the remote cloud.

Structured content mode

curl -X "POST" "https://p1.c2.exosite.io"
     -H 'Authorization: Bearer H4sj8fbPFx86toDR'
     -H 'Content-Type: application/cloudevents+json; charset=utf-8'
     -H 'WebHook-Request-Origin: exosite.cloud.test'
     -H 'Host: p1.c2.exosite.io'
     -d '{
          "specversion": "1.0",
          "id": "bf7e2043-0c3f-4727-aac3-cdad95d245e8",
          "source": "/remote-cloud/application/devices",
          "time": "2022-06-29T13:27:15+02:00",
          "type": "exosite.identity.deleted",
          "subject": "00001"
        }'

Binary content mode

curl -X "POST" "https://p1.c2.exosite.io"
     -H 'Authorization: Bearer H4sj8fbPFx86toDR'
     -H 'ce-specversion: 1.0'
     -H 'ce-id: 3770399b-f414-44c9-bc43-6176a5049abe'
     -H 'ce-source: /remote-cloud/application/devices'
     -H 'ce-time: 2022-06-29T12:06:44+00:00'
     -H 'ce-type: exosite.identity.deleted'
     -H 'ce-subject: 00001'
     -H 'WebHook-Request-Origin: exosite.cloud.test'
     -H 'Host: p1.c2.exosite.io'

Field definitions:

subject: The identity of the device having been deleted.
type: exosite.identity.deleted

Actions taken by the IoT Connector:

If the device exists in the IoT Connector:

The identity identified by subject is removed from the IoT Connector
The deleted event is generated in the platform.

If the device does not exist in the IoT Connector:

No action is taken.

exosite.identity.connected¶

This event is sent when a new device has just connected to the remote cloud.

Structured content mode

curl -X "POST" "https://p1.c2.exosite.io"
     -H 'Authorization: Bearer H4sj8fbPFx86toDR'
     -H 'Content-Type: application/cloudevents+json; charset=utf-8'
     -H 'WebHook-Request-Origin: exosite.cloud.test'
     -H 'Host: p1.c2.exosite.io'
     -d '{
          "specversion": "1.0",
          "id": "bf7e2043-0c3f-4727-aac3-cdad95d245e8",
          "source": "/remote-cloud/application/devices",
          "time": "2022-06-29T13:27:15+02:00",
          "type": "exosite.identity.connected",
          "subject": "00001"
        }'

Binary content mode

curl -X "POST" "https://p1.c2.exosite.io"
     -H 'Authorization: Bearer H4sj8fbPFx86toDR'
     -H 'ce-specversion: 1.0'
     -H 'ce-id: 3770399b-f414-44c9-bc43-6176a5049abe'
     -H 'ce-source: /remote-cloud/application/devices'
     -H 'ce-time: 2022-06-29T12:06:44+00:00'
     -H 'ce-type: exosite.identity.connected'
     -H 'ce-subject: 00001'
     -H 'WebHook-Request-Origin: exosite.cloud.test'
     -H 'Host: p1.c2.exosite.io'

Field definitions:

subject: The identity of the device that has just connected.
type: exosite.identity.connected

Actions taken by the IoT Connector:

If auto-provisioning is enabled, on the first ever connection:

An identity is created in the IoT Connector
The identity is placed in provisioned status
The provisioned event is generated in the platform.

On all connections:

The connect event is generated in the platform.

exosite.identity.disconnected¶

This event is sent when a new device has just disconnected from the remote cloud.

Structured content mode

curl -X "POST" "https://p1.c2.exosite.io"
     -H 'Authorization: Bearer H4sj8fbPFx86toDR'
     -H 'Content-Type: application/cloudevents+json; charset=utf-8'
     -H 'WebHook-Request-Origin: exosite.cloud.test'
     -H 'Host: p1.c2.exosite.io'
     -d '{
          "specversion": "1.0",
          "id": "bf7e2043-0c3f-4727-aac3-cdad95d245e8",
          "source": "/remote-cloud/application/devices",
          "time": "2022-06-29T13:27:15+02:00",
          "type": "exosite.identity.disconnected",
          "subject": "00001"
        }'

Binary content mode

curl -X "POST" "https://p1.c2.exosite.io"
     -H 'Authorization: Bearer H4sj8fbPFx86toDR'
     -H 'ce-specversion: 1.0'
     -H 'ce-id: 3770399b-f414-44c9-bc43-6176a5049abe'
     -H 'ce-source: /remote-cloud/application/devices'
     -H 'ce-time: 2022-06-29T12:06:44+00:00'
     -H 'ce-type: exosite.identity.disconnected'
     -H 'ce-subject: 00001'
     -H 'WebHook-Request-Origin: exosite.cloud.test'
     -H 'Host: p1.c2.exosite.io'

Field definitions:

subject: The identity of the device that has just disconnected.

Actions taken by the IoT Connector:

If the device exists in the IoT Connector:

The disconnect event is generated in the platform.

If the device does not exist in the IoT Connector:

No action is taken.