Device HTTP API¶
The purpose of this API is to provide device software and firmware with the ability to communicate with Murano. Devices use this API to perform actions such as authentication, posting sensor and status data, and retrieving configuration and control updates.
IMPORTANT: Your hardware & Library MUST be compatible with the device API connectivity TLS requirements , make sure you read it first.
Note: When used in this document, "timestamp" is a Unix timestamp, defined as the number of seconds that have elapsed since 00:00:00 Coordinated Universal Time \(UTC\), Thursday, 1 January 1970.
General¶
HTTP Responses ¶
Typical HTTP response codes include:
Code | Response | Description |
---|---|---|
200 | OK | Successful request, returning requested values |
204 | No Content | Successful request, nothing will be returned |
4xx | Client Error | There was an error* with the request by the client |
401 | Unauthorized | Missing or invalid credentials |
5xx | Server Error | Unhandled server error. Contact support. |
Note: Aliases that are not found are not considered errors in the request. See the documentation for write, read, and Hybrid Read Write for details.
Libraries and Sample Code ¶
Sample code, libraries, and client applications that uses this API can be found on the Device API Libraries / Reference page.
Notational Conventions ¶
This document uses the following notational conventions:
- A name in angle brackets \(e.g.,
<myvar>
is a placeholder that will be defined elsewhere\) ...
represents one or more of the previous items- Curly brackets around HTTP headers represent optional or conditional headers.
Encoding ¶
As of Content-Type: application/x-www-form-urlencoded
, non-alphanumeric characters from both part of the equal sign must be percent encoded as defined by the standard https://www.w3.org/TR/html401/interact/forms.html.
The Content-Encoding
and Accept-Encoding
headers as defined by the RFC 2616 standard, are supported. The following encoding types are implemented:
gzip
: A format using the Lempel-Ziv coding (LZ77), with a 32-bit CRC.identity
: No encoding is applied. This is the default when headers are omitted.
Note, that gzip
encoding for the response (Accept-Encoding
) is not supported on the /provision/download
endpoint.
If the caller specifies a value for Accept-Encoding
or Content-Encoding
that is not supported by Murano, a 406
or 415
response will be returned, respectively.
gzip Encoding Examples
Below is an example of writing a gzip'ed value. The value will be unzipped before being applied to the resource.
Request
POST /onep:v1/stack/alias HTTP/1.1
Host: <IOT_CONNECTOR_FQDN>
{X-Exosite-CIK: <token>}
{Authorization: Basic <base64-encoding of identity:password>}
Content-Type: application/x-www-form-urlencoded; charset=utf-8
Content-Encoding: gzip
Content-Length: <length>
<blank_line>
<gzipped 'data_in={"001":73.16492}'>
Response
HTTP/1.1 204 No Content
Date: <date>
Server: Murano
Connection: Keep-Alive
<blank_line>
Here is an example of reading a resource, specifying that gzip
and identity
encodings are accepted, with gzip
being the preferred one:
Request
GET /onep:v1/stack/alias?<alias_1> HTTP/1.1
Host: <IOT_CONNECTOR_FQDN>
{X-Exosite-CIK: <token>}
{Authorization: Basic <base64-encoding of identity:password>}
Accept: application/x-www-form-urlencoded; charset=utf-8
Accept-Encoding: identity;q=0.5, gzip;q=1.0
<blank_line>
Response
HTTP/1.1 200 OK
Date: <date>
Server: Murano
Connection: Close
Content-Length: <length>
Content-Encoding: gzip
<blank_line>
<gzipped 'alias_1=value_1'>
Product Domain ¶
Each Murano Product is assigned a unique API domain for devices to connect to. In this document, we will use the example domain of <IOT_CONNECTOR_FQDN>
as an example Product domain. Replace the example domain with the Endpoint shown in the associated Product page of your account UI.
TLS Connection Requirements ¶
When making a secure TLS connection attempt to the API domain, it is required to specify the domain as the SNI field in the TLS connection request. The HTTP "Host" header must also be set to the domain name. Any mismatch or use of an invalid domain will result in the connection being terminated without response. Also read the Device API connectivity TLS Specs..
Bandwidth considerations ¶
TLS handshake has a significant request size and latency overhead for small HTTP payloads. This is because the server needs to provide the certificate chain \(typically 3~4kb\) to the device for verification. HTTP keep-alive can be used to mitigate most of the TLS overhead, especially when a device sends multiple small requests.
From our tests, sending three sequential write requests \(single resource\) without HTTP keep-alive used ~16kb bandwidth. The same requests with HTTP keep-alive enabled, over a single connection, used ~6kb bandwidth and had %40 lower latency.
Restrictions ¶
There is no specific bandwidth or message size limitation, however each individual resource size MUST respect the maximum size of 1048576 bytes.
Authentication ¶
A device may authenticate with a TLS Client Certificate, by using a secret Token specified in the "X-Exosite-CIK" HTTP request header or by using a Password specified in the "Authorization" HTTP request header.
Murano Products can be configured to allow devices to provision their own identities or to allow only whitelisted device identities to be provisioned. This configuration option is available via the Product Settings tab as the Allow devices to register their own identity checkbox. When checked \(default\), any device using this API with the assigned Product Endpoint can provision themselves without the need to be whitelisted in advance. If this option is disabled \(unchecked\), then device identities must be whitelisted in advance in order to provision and communicate with the Product.
NOTE: When using a TLS Client Certificate for authentication, the certificate "Subject" CommonName \(CN\) must hold the connecting device's identity.
Provisioning and OTA updates ¶
To connect your device remember to download and install the Murano Root server CA. Otherwise your firmware will probably raise a security error during connection.
For development on your operating system, use the Digicert Root CA which is already known by your system. \(So no need of downloading and installing it manually.\)
Provision ¶
When a Product is configured for Token authentication, calling this endpoint will provision credentials for the given <identity>
and return the secret <token>
that the device must use for all subsequent API requests. Identities must also conform to the identity format specification defined in the Settings tab of the Product UI.
When a Product is configured for Password authentication, calling this endpoint will provision credentials for the given <identity>
and <password>
\(at least 20 characters\) that the device must use for all subsequent API requests. Identities must also conform to the identity format specification defined in the Settings tab of the Product UI.
When a Product is configured for Certificate authentication, provisioning will happen automatically when the device first connects. The request the device will make on its initial connection, is irrelevant and is not part of the provisioning process. That said, the initial connection results in two subsequent steps but separate operations:
- provisioning,
- the actual request.
The following behavior can be observed:
- If
step 1
fails, the connection will be terminated andstep 2
will not be attempted. - If
step 1
succeeds,step 2
will be attempted which may still fail. The response will depend on the outcome ofstep 2
. - If the request is made to the "activate" \(described below\) endpoint, it will act as a re-provisioning request since provisioning will already have happened automatically.
NOTE: After that, the device must use the provisioned credentials to authenticate all subsequent API requests \(e.g., read, write and ...\).
The request takes the following form. The Content-Type and body will depend on the type of authentication being provisioned.
Token and CIK auth ¶
POST /provision/activate HTTP/1.1
Host: <IOT_CONNECTOR_FQDN>
Content-Type: application/x-www-form-urlencoded; charset=utf-8
Content-Length: <length>
id=<identity>
Response
HTTP/1.1 200 OK
Date: <date>
Server: Murano
Connection: Keep-Alive
Content-Length: <length>
Content-Type: text/plain; charset=utf-8
<token>
The response should return a 40-character string token also called the CIK.
Provision identity 12345678
$ curl -i 'https://<IOT_CONNECTOR_FQDN>/provision/activate' \
-H 'Content-Type: application/x-www-form-urlencoded; charset=utf-8' \
-d 'id=12345678'
Response
HTTP/1.1 200 OK
Date: Fri, 05 May 2017 00:11:22 GMT
Server: Murano
Connection: Keep-Alive
Content-Length: 40
Content-Type: text/plain; charset=utf-8
22596363b3de40b06f981fb85d82312e8c0ed511
Provision identity 12345678
import json
import requests
requests.post(
"https://<IOT_CONNECTOR_FQDN>/provision/activate",
headers={ "Content-Type": "application/x-www-form-urlencoded; charset=utf-8" },
data={ "id": "12345678" }
)
Response
HTTP/1.1 200 OK
Date: Fri, 05 May 2017 00:11:22 GMT
Server: Murano
Connection: Keep-Alive
Content-Length: 40
Content-Type: text/plain; charset=utf-8
22596363b3de40b06f981fb85d82312e8c0ed511
Password auth ¶
POST /provision/activate HTTP/1.1
Host: <IOT_CONNECTOR_FQDN>
Content-Type: application/x-www-form-urlencoded; charset=utf-8
Content-Length: <length>
id=<identity>
password=<password>
Response
HTTP/1.1 204 OK
Date: <date>
Server: Murano
Connection: Keep-Alive
Provision Responses¶
Provision specific response codes include:
Code | Response | Description |
---|---|---|
200 | OK | Successful request, returning TOKEN |
204 | No Content | Successful request, nothing will be returned |
404 | Not Found | Whitelisting is required and the <identity> is not whitelisted |
409 | Conflict | The <identity> has already been provisioned |
415 | Unsupported Media Type | The provided <password> is less than 20 characters |
Once activated a devices will be visible on the Murano IoT-Connector devices page with its IP set.
Reprovision ¶
The idea of reprovisioning is that, in case device credentials get compromised or for other reasons, new credentials need to be set for a device already existing in the database. The reprovisioning process is the same as provisioning but requires that the device authenticate first. So the "activate" endpoint described above is called the same way but with authentication credentials provided in the call.
That said, for certificate authentication, re-provisioning takes a slightly different form in that the new certificate must be sent in the payload, as follows:
Certificate auth ¶
POST /provision/activate HTTP/1.1
Host: <IOT_CONNECTOR_FQDN>
Content-Type: application/x-pem-file; charset=utf-8
Content-Length: <length>
<certificate_in_PEM_format>
Response
HTTP/1.1 204 OK
Date: <date>
Server: Murano
Connection: Keep-Alive
Writing And Reading Data ¶
Tip
See the ExoSense Data IO Schema for more context about the values used in the examples below.
Write ¶
Write one or more resources identified by <alias>
with the given <value>
. The connecting device is identified and authenticated with the provided <token>
, <password>
or <certificate>
. If the Murano Product has defined resources with matching <alias>
es, the <value>
s are stored as the device state at timestamp the data was received by Murano. If multiple aliases are specified, they are written at the same timestamp. This method generates an event type of "data_in" on the Murano Device2 service and may be used to, for instance, store the data in the Tsdb database or alert users or other services of the event.
Request
POST /onep:v1/stack/alias HTTP/1.1
Host: <IOT_CONNECTOR_FQDN>
{X-Exosite-CIK: <token>}
{Authorization: Basic <base64-encoding of identity:password>}
Content-Type: application/x-www-form-urlencoded; charset=utf-8
Content-Length: <length>
<blank_line>
<alias_1>=<value_1>&<alias_2...>=<value_2...>&<alias_n>=<value_n>
Response
HTTP/1.1 204 No Content
Date: <date>
Server: Murano
Connection: Keep-Alive
<blank_line>
- See HTTP Responses for a full list of responses.
Write Example
$ curl -i 'https://<IOT_CONNECTOR_FQDN>/onep:v1/stack/alias' \
-H 'X-Exosite-CIK: 22596363b3de40b06f981fb85d82312e8c0ed511' \
-d 'data_in={"001":73.16492}'
import json
import requests
requests.post(
f"https://<IOT_CONNECTOR_FQDN>/onep:v1/stack/alias",
headers={
"Content-Type": "application/x-www-form-urlencoded; charset=utf-8",
"X-Exosite-CIK": "<TOKEN>"
},
data={
"config_io": json.dumps({"channels":{"001":{"display_name":"Temperature","description":"Ambient tmperature reading","properties":{"data_type":"TEMPERATURE","data_unit":"DEG_FAHRENHEIT","precision":2},"protocol_config":{"report_rate":30000,"timeout":60000}}}}),
"data_in": json.dumps({"001":73.16492})
}
)
Response
HTTP/1.1 204 No Content
Date: Fri, 05 May 2017 00:11:22 GMT
Server: Murano
Connection: Keep-Alive
Read ¶
Read the most recent value from one or more resources. If at least one <alias>
resource definition exists, the set value will be returned.
Read the most recent value from one or more resources with <alias>
. The client \(e.g., device or portal\) to read from is identified by <token>
, <password>
or <certificate>
. If at least one <alias>
is found and has data, data will be returned.
Request
GET /onep:v1/stack/alias?<alias_1>&<alias_2...>&<alias_n> HTTP/1.1
Host: <IOT_CONNECTOR_FQDN>
{X-Exosite-CIK: <token>}
{Authorization: Basic <base64-encoding of identity:password>}
Accept: application/x-www-form-urlencoded; charset=utf-8
<blank_line>
Response
HTTP/1.1 200 OK
Date: <date>
Server: Murano
Connection: Close
Content-Length: <length>
<blank_line>
<alias_1>=<value_1>&<alias_2...>=<value_2...>&<alias_n>=<value_n>
- Response may also be
HTTP/1.1 204 No Content
if either none of the aliases are found or the device state for the given aliases are empty - See HTTP Responses for a full list of responses
Read Example
$ curl -i 'https://<IOT_CONNECTOR_FQDN>/onep:v1/stack/alias?config_io' \
-H 'X-Exosite-CIK: 22596363b3de40b06f981fb85d82312e8c0ed511' \
-H 'Accept: application/x-www-form-urlencoded; charset=utf-8'
Response
HTTP/1.1 200 OK
Date: Fri, 05 May 2017 00:11:22 GMT
Server: Murano
Connection: Keep-Alive
Content-Length: 13
Content-Type: application/x-www-form-urlencoded; charset=utf-8
config_io=%7b%22channels%22%3a+%7b%22001%22%3a+%7b%22description%22%3a+%22URL+string%22%2c+%22display_name%22%3a+%22Raw+URL+String%22%2c+%22properties%22%3a+%7b%22data_type%22%3a+%22STRING%22%7d%2c+%22report_rate%22%3a+10000%2c+%22sample_rate%22%3a+10000%7d%7d%7d
Hybrid Read Write ¶
Write one or more values with <alias_w>
with given <value>
and also read the most recent values from <alias_r>
.
Note: All writes occur before all reads.
Request
POST /onep:v1/stack/alias?<alias_r1>&<alias_r2...>&<alias_rn> HTTP/1.1
Host: <IOT_CONNECTOR_FQDN>
{X-Exosite-CIK: <token>}
{Authorization: Basic <base64-encoding of identity:password>}
Accept: application/x-www-form-urlencoded; charset=utf-8
Content-Type: application/x-www-form-urlencoded; charset=utf-8
Content-Length: <length>
<blank_line>
<alias_w1>=<value_1>&<alias_w2...>=<value_2...>&<alias_wn>=<value_n>
Response
HTTP/1.1 200 OK
Date: <date>
Server: Murano
Connection: Close
Content-Length: <length>
<blank_line>
<alias_r1>=<value_1>&<alias_r2...>=<value_2...>&<alias_rn>=<value_n>
- Response may also be
HTTP/1.1 204 No Content
if either none of the aliases are found or the device state for the given aliases are empty - See HTTP Responses for a full list of responses
Hybrid Read Write Example
$ curl 'https://<IOT_CONNECTOR_FQDN>/onep:v1/stack/alias?config_io' \
-H 'X-Exosite-CIK: 22596363b3de40b06f981fb85d82312e8c0ed511' \
-H 'Accept: application/x-www-form-urlencoded; charset=utf-8' \
-H "Content-Type: application/x-www-form-urlencoded; charset=utf-8" \
-d 'data_in={"001":73.16492}'
HTTP/1.1 200 OK
Date: Fri, 05 May 2017 00:11:22 GMT
Server: Murano
Connection: Keep-Alive
Content-Length: 13
Content-Type: application/x-www-form-urlencoded; charset=utf-8
config_io=%7b%22channels%22%3a+%7b%22001%22%3a+%7b%22description%22%3a+%22URL+string%22%2c+%22display_name%22%3a+%22Raw+URL+String%22%2c+%22properties%22%3a+%7b%22data_type%22%3a+%22STRING%22%7d%2c+%22report_rate%22%3a+10000%2c+%22sample_rate%22%3a+10000%7d%7d%7d
Long Polling ¶
The read endpoint also supports long polling. Long polling is a method of getting a server push without the complexities of setting up publicly accessible HTTP server endpoints on your device. As the name suggests, long polling is similar to normal polling of an HTTP resource, but instead of requiring the client to make a new request to the server constantly, the server will wait to return until it has new information to return to the client \(or a timeout has been reached\).
To perform a request with long polling, simply add the header Request-Timeout: <milliseconds>
to your request. The server will then wait until a new datapoint is written to the given resource and will then immediately return the value. If no datapoint is written before that time, a 304 Not Modified
is returned and the client may make another long polling request to continue monitoring that resource.
You may also optionally add an If-Modified-Since
header to specify a start time to wait. This is exactly the same as the alias.last
semantics in scripting. You will want to use this if it's important that you receive all updates to a given resource; otherwise it is possible to miss points written between long polling requests.
Note: Only one resource may be read at a time when using long polling.
Request
GET /onep:v1/stack/alias?<alias_1> HTTP/1.1
Host: <IOT_CONNECTOR_FQDN>
{X-Exosite-CIK: <token>}
{Authorization: Basic <base64-encoding_of_identity:password>}
Accept: application/x-www-form-urlencoded; charset=utf-8
Request-Timeout: <timeout>
If-Modified-Since: <timestamp>
<blank_line>
<alias>
is the alias you monitor for new datapoints.Request-Timeout
specifies how long to wait on changes.<timeout>
is a millisecond value and cannot be more than 300 seconds \(300,000 ms\).If-Modified-Since
specifies waiting on aliases since the<timestamp>
.<timestamp>
can be timestamp seconds since 1970-01-01 00:00:00 UTC or standard <a href=http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html>HTTP-Date format. If this is not specified, it defaults to "now."
Response
When the device set state is updated:
HTTP/1.1 200 OK
Date: <date>
Server: Murano
Connection: Close
Content-Length: <length>
Content-Type: application/x-www-form-urlencoded; charset=utf-8
Last-Modified: <value-set-date>
<blank_line>
<alias>=<value>
If the device state is not updated before timeout:
HTTP/1.1 304 Not Modified
Date: <date>
Server: Murano
Connection: Close
Content-Length: <length>
<blank_line>
When the resource is written and a value is returned, a Last-Modified
header is included. When it is vital for your application to receive all updates to a resource, you can pass the Last-Modified
header value back as the If-Not-Modified-Since
header in your next request to make sure you do not miss any points that may have been written since the last request returned.
Long Polling Example
$ curl 'https://<IOT_CONNECTOR_FQDN>/onep:v1/stack/alias?config_io' \
-H 'X-Exosite-CIK: 22596363b3de40b06f981fb85d82312e8c0ed511' \
-H 'Accept: application/x-www-form-urlencoded; charset=utf-8' \
-H 'Request-Timeout: 30000' \
-H 'If-Modified-Since: 1408088308'
HTTP/1.1 200 OK
Date: Fri, 05 May 2017 00:11:22 GMT
Server: Murano
Connection: Keep-Alive
Content-Length: 13
Content-Type: application/x-www-form-urlencoded; charset=utf-8
Last-Modified: 1408088412
config_io=%7b%22channels%22%3a+%7b%22001%22%3a+%7b%22description%22%3a+%22URL+string%22%2c+%22display_name%22%3a+%22Raw+URL+String%22%2c+%22properties%22%3a+%7b%22data_type%22%3a+%22STRING%22%7d%2c+%22report_rate%22%3a+10000%2c+%22sample_rate%22%3a+10000%7d%7d%7d
Record ¶
Report data to one or more historical timestamps of one or more aliases. If the Murano Product has defined resources with matching <alias>
es, the most recent <value>
s are stored as the device state with the timestamp the data was received by Murano. The connecting device is identified and authenticated with the provided <token>
.
Timestamp support floating-point based on seconds and interpret to microseconds.
timestamp1=1551674686.339 s -> 1551674686339000 us
timestamp2=1551674686.339876 s -> 1551674686339876 us
Acceptable values for timestamps are as follows:
- when positive, the value is the number of seconds since 1970.01.01 00:00:00 UTC.
- when negative, the value is a time offset in seconds relative to 'now'. For example, -3600 means one hour ago.
Request
This example records to <alias_1>
in <timestamp_1>
and <timestamp_2>
, and records to <alias_2>
in <timestamp_3>
and <timestamp_4>
.
POST /onep:v1/stack/record HTTP/1.1
Host: <IOT_CONNECTOR_FQDN>
{X-Exosite-CIK: <token>}
{Authorization: Basic <base64-encoding of identity:password>}
Content-Type: application/x-www-form-urlencoded; charset=utf-8
Content-Length: <length>
<blank_line>
alias=<alias_1>&<timestamp_1>=<value_1>&<timestamp_2>=<value_2>&alias=<alias_2>&<timestamp_3>=<value_3>&<timestamp_4>=<value_4>
Response
HTTP/1.1 204 No Content
Date: <date>
Server: <server>
Connection: Close
Content-Length: 0
<blank_line>
- See HTTP Responses for a full list of responses.
- If
<timestamp_1>
and<timestamp_2>
,<timestamp_3>
and<timestamp_4>
have less than one-second difference:
HTTP/1.1 409 Conflict
Date: <date>
Server: <server>
Connection: Close
Content-Length: <length>
<alias_1>=<timestamp_1>&<alias_2>=<timestamp_2>
Record Example
Report three values to the same alias, data_in
, using each supported timestamp format:
$ curl https://<IOT_CONNECTOR_FQDN>/onep:v1/stack/record \
-H 'X-Exosite-CIK: 22596363b3de40b06f981fb85d82312e8c0ed511' \
-H 'Content-Type: application/x-www-form-urlencoded; charset=utf-8' \
-d 'alias=data_in&1506708551={"001":73.16492}&1506708552.127={"001":74.78193}&1506708553.264391={"001":75.35915}'
Static Content Files ¶
This chapter explains how to interact with static files provided by Murano's Content micro-service.
List Available Content ¶
List available content <content-id>
s.
GET /provision/download HTTP/1.1
Host: <IOT_CONNECTOR_FQDN>
{X-Exosite-CIK: <token>}
{Authorization: Basic <base64-encoding_of_identity:password>}
<blank_line>
Response
HTTP/1.1 200 OK
Date: <date>
Server: Murano
Connection: Keep-Alive
Content-Length: <length>
Content-Type: text/csv; charset=utf-8
<content-id_1>
<content-id_2...>
<content-id_n>
Response may also be:
- See HTTP Responses for a full list of responses
List Available Content Example
$ curl -i 'https://<IOT_CONNECTOR_FQDN>/provision/download' \
-H 'X-Exosite-CIK: 22596363b3de40b06f981fb85d82312e8c0ed511'
HTTP/1.1 200 OK
Date: Fri, 05 May 2017 00:11:22 GMT
Server: Murano
Connection: Keep-Alive
Content-Length: 71
Content-Type: text/csv; charset=utf-8
MANIFEST
fw_20150519_rev01b.bin
fw_20160101_rev02.bin
splash01.png
splash02.png
Get Content Info ¶
Retrieve meta information \(content-type, size, updated timestamp, description\) for the specified <content-id>
.
GET /provision/download?id=<content-id>&info=true HTTP/1.1
Host: <IOT_CONNECTOR_FQDN>
{X-Exosite-CIK: <token>}
{Authorization: Basic <base64-encoding of identity:password>}
<blank_line>
Response
HTTP/1.1 200 OK
Date: <date>
Server: Murano
Connection: Keep-Alive
Content-Length: <length>
Content-Type: text/csv; charset=utf-8
<content-type>,<byte-size>,<updated-timestamp>,<description>
Response may also be:
- See HTTP Responses for a full list of responses
Get Content Info Example
$ curl -i 'https://<IOT_CONNECTOR_FQDN>/provision/download?id=splash01.png&info=true' \
-H 'X-Exosite-CIK: 22596363b3de40b06f981fb85d82312e8c0ed511'
HTTP/1.1 200 OK
Date: Fri, 05 May 2017 00:11:22 GMT
Server: Murano
Connection: Keep-Alive
Content-Length: 45
Content-Type: text/csv; charset=utf-8
image/png,23427,1462500951,Boot splash screen
Download Content ¶
Download the content <content-id>
in full or in part. To request chunks of the content, use the header Range: bytes=<range-specifier>
. <range-specifier>
takes the form of X-Y
where both X
and Y
are optional but at least one of them must be present. X
is the start byte position to return. Y
is the end position. Both are 0 based. If X
is omitted, Y
will request the last Y
count of bytes of the content. If Y
is omitted, it will default to the end of the content. The response Content-Type
header will be as defined in the content meta info.
GET /provision/download?id=<content-id> HTTP/1.1
Host: <IOT_CONNECTOR_FQDN>
{X-Exosite-CIK: <token>}
{Authorization: Basic <base64-encoding of identity:password>}
{Range: bytes=<range-specifier>}
<blank_line>
Response
HTTP/1.1 200 OK
Date: <date>
Server: Murano
Connection: Keep-Alive
Content-Length: <number of bytes being returned>
Content-Type: <content-type>
Transfer-Encoding: chunked
{Accept-Ranges: bytes}
{Content-Range: bytes <first_position>-<last_position>/<total_length>}
<blob>
Response may also be:
HTTP/1.1 206 Partial Content
if the response is partial.- See HTTP Responses for a full list of responses
Download Content Example
Download entire file
$ curl -i 'https://<IOT_CONNECTOR_FQDN>/provision/download?id=splash01.png' \
-H 'X-Exosite-CIK: 22596363b3de40b06f981fb85d82312e8c0ed511'
HTTP/1.1 200 OK
Date: Fri, 05 May 2017 00:11:22 GMT
Server: Murano
Connection: Keep-Alive
Content-Length: 23427
Content-Type: image/png
Transfer-Encoding: chunked
<23427-byte-blob>
Download first 1024 bytes of the file
$ curl -i -r 0-1023 https://<IOT_CONNECTOR_FQDN>/provision/download?id=splash01.png \
-H 'X-Exosite-CIK: 22596363b3de40b06f981fb85d82312e8c0ed511'
HTTP/1.1 200 OK
Date: Fri, 05 May 2017 00:11:22 GMT
Server: Murano
Connection: Keep-Alive
Content-Length: 1024
Content-Type: image/png
Transfer-Encoding: chunked
Accept-Ranges: bytes
Content-Range: bytes 0-1023/23427
<1024-byte-blob>
Upload Content ¶
The device API provides the ability to upload files from the device into Murano Content Store. Uploaded files will be accessible on the Content Service under the device2/<device identity-id>/
folder. Files uploaded by a given device will be available for download to that device only and will not be available to other devices. This method generates an event type of "content_in" on the Murano Device2 service. It carries all the meta information about the uploaded content but not the actual content blob. See the device2 service reference.
PUT /onep:v1/content/<content-id>?tags=<tags> HTTP/1.1
Host: <IOT_CONNECTOR_FQDN>
{X-Exosite-CIK: <token>}
{Authorization: Basic <base64-encoding of identity:password>}
Content-Length: <length in bytes>
{Content-Type: <media type of file>}
{Content-MD5: <base64-encoding of MD5 sum value>}
<blob as body payload>
is the url encoding of a JSON key/value object. Eg. {"mytag1":"myvalue1","mytag2":"myvalue2"}
Response
HTTP/1.1 204 No Content
Date: <date>
Server: Murano
Connection: Keep-Alive
Upload Content Example
Using Curl command tool to upload a file.
$ curl -i -X PUT https://<IOT_CONNECTOR_FQDN>/onep:v1/content/logs/error.log?tags=%7B%22abc%22%3A123%2C%22xyz%22%3A456%7D \
-H 'Content-Type: text/plain' \
-H 'Content-Length: 13' \
-H 'Content-MD5: 7naXGYTSUTihmayQVTQB3Q==' \
-H 'X-Exosite-CIK: 0123456789012345678901234567890123456789' \
-d @error.log
Python example to upload a file.
import os
import json
import requests
import base64
import hashlib
import urllib.parse
def checksum(filename, hash_factory=hashlib.md5, chunk_num_blocks=128):
h = hash_factory()
with open(filename,'rb') as f:
for chunk in iter(lambda: f.read(chunk_num_blocks*h.block_size), b''):
h.update(chunk)
return h.digest()
connector_fqdn = "<IOT_CONNECTOR_FQDN>"
token = "<TOKEN>"
file_name = "myfile.csv"
file_path = f"./{file_name}"
encoded_name = urllib.parse.quote(file_name)
upload_url = f"https://{connector_fqdn}/onep:v1/content/{encoded_name}"
checksum = checksum(file_path)
checksum = base64.b64encode(checksum).decode('utf-8')
content_length = os.path.getsize(file_path)
headers = {
'Content-Type': 'application/x-www-form-urlencoded',
'X-Exosite-CIK': token,
'Content-Type': 'text/csv',
'Content-Length': str(content_length),
'Content-MD5': checksum
}
# Streaming uploads, which allow you to send large files or streams without reading into memory
with open(file_path, 'rb') as file:
r = requests.put(
upload_url,
headers=headers,
data=file)
print(r, r.text)
Response
HTTP/1.1 204 OK
Date: Tue, 02 Jul 2019 15:14:22 GMT
Server: Murano
Connection: Keep-Alive
Upload Content in Parts¶
If content is smaller than 100MB in size, it can be uploaded in a single push, as described in (Upload Content). For files over 100MB, Murano provides a way to upload them in multiple parts. Such multi-part upload involves three steps:
- The initiation of the upload
- The uploading of one or more part
- The optional step of completing the upload
An upload starts by sending a request to initiate the upload, specifying a content-id, an optional length and optional tags. This creates a placeholder for the actual content blob to be uploaded to.
The second step is sending one or more request, each containing a separate part of the content blob with a range specifier so that Murano can assemble all parts into one whole. The parts don’t need to be in sequential order. For each part, an MD5 sum can be specified which, if provided, Murano will validate.
If an upload attempt specifies a range that overlaps that of another part already uploaded, the request will fail.
Each part must be exactly 5MB (5 * 1024 * 1024) in size except the last one which can be smaller.
If length was given in the initial request, the upload completes automatically when a contiguous range of as many bytes have been uploaded. If length was not initially specified, the upload must be finalized by sending a 0-byte part.
If any of the requests, including initiation, part upload and finalization, fails, the request can be retried. Also, the initiation and part upload requests are idempotent - they can be repeated without side effects - except for the final part (if it automatically completes the upload) and the optional 0-byte finalization request. Once an upload has been finalized, attempts to re-upload parts of it or re-finalize it, will fail.
If a finalization attempt is made when not enough bytes have been uploaded or there is a gap in the uploaded ranges, the finalization will fail.
Status information about content that is in the process of being uploaded, can be obtained through the 'Get Content Info' endpoint. It will return range and status information about the parts that have already been, or are being, uploaded.
Once successfully finalized, content uploaded in parts will show up as just another content item in the device2/<identity>/<content-id>
area and a content_in
event is triggered.
Below are the actual requests involved in the process.
Initiate an upload
POST /onep:v1/content/ HTTP/1.1
Host: <IOT_CONNECTOR_FQDN>
{X-Exosite-CIK: <token>}
{Authorization: Basic <base64-encoding of identity:password>}
Content-Length: <length in bytes>
{Content-Type: application/json}
{
"id":<content-id>,
"length":<byte-count>,
"type":<media type of file>,
"tags":{"mytag1":"myvalue1","mytag2":"myvalue2"}
}
Response
HTTP/1.1 204 No Content
Date: <date>
Server: Murano
Connection: Keep-Alive
Upload a part
PATCH /onep:v1/content/<content-id> HTTP/1.1
Host: <IOT_CONNECTOR_FQDN>
{X-Exosite-CIK: <token>}
{Authorization: Basic <base64-encoding of identity:password>}
Content-Length: <length in bytes>
MD5: <base-64 encoded MD5 sum value of payload>
{Content-Type: application/octet-stream}
Content-Range: bytes <first byte position>-<last byte position>/<total size or *>
<blob as body payload>
Response
HTTP/1.1 204 No Content
Date: <date>
Server: Murano
Connection: Keep-Alive
Finalize an upload
PATCH /onep:v1/content/<content-id> HTTP/1.1
Host: <IOT_CONNECTOR_FQDN>
{X-Exosite-CIK: <token>}
{Authorization: Basic <base64-encoding of identity:password>}
Content-Length: 0
no body payload
Response
HTTP/1.1 204 No Content
Date: <date>
Server: Murano
Connection: Keep-Alive
Get Upload Info
GET /onep:v1/content/<content-id> HTTP/1.1
Host: <IOT_CONNECTOR_FQDN>
{X-Exosite-CIK: <token>}
{Authorization: Basic <base64-encoding of identity:password>}
Content-Length: 0
no body payload
Response
HTTP/1.1 200 OK
Date: <date>
Server: Murano
Connection: Close
Content-Length: <length>
<blank_line>
{
"id":"device2/<identity>/<content-id>",
"incomplete_parts":[[12000000,17999999]],
"length":25000000,
"ranges":[[0,5999999],[6000000,11999999]],
"type":"text/plain"
}
Debug events
Besides the content_in
event (device2 murano service) indicating a successful upload, to aid development debug events are sent to the product debug web socket endpoint as follows:
body_size_too_large
: the part was greater than 100MB in sizecontent_notfound
: the content identified by content-id in a part upload or finalization attempt was not foundincomplete_part
: an attempt was made to finalize an upload with not enough bytes uploaded or gap in the uploaded rangesincomplete_range
: the Content-Range header specified a range longer than bytes provided in the bodyinvalid_content_id
: the content id is somehow invalidinvalid_range
: the Content-Range header was invalidinvalid_tags
: the tags included with the initiation request was not proper JSON or not key/value pairsinvalid_md5
: the provided MD5 header contained a value that did not successfully validate for the body payloadoverlapping_range
: a part was attempted to be uploaded with Content-Range overlapping a previously uploaded partupload_part_too_small
: the part being uploaded was less than 5MB in sizeinternal
: an error internal to Murano occurred
Content over MQTT port¶
If your device is set to use MQTT connection you can still interact with content files using the HTTP API over the same port \(eg. 8883\). However client MUST set the ALPN header to enforce server-side protocol selection. Some client such as Curl send ALPN by default and nothing need to be done.
Example
File upload over MQTT port using Python.
import os
import json
import http.client as client
import base64
import hashlib
import urllib.parse
import ssl
def checksum(filename, hash_factory=hashlib.md5, chunk_num_blocks=128):
h = hash_factory()
with open(filename,'rb') as f:
for chunk in iter(lambda: f.read(chunk_num_blocks*h.block_size), b''):
h.update(chunk)
return h.digest()
def ssl_alpn():
try:
print(f"OpenSSL version:{ssl.OPENSSL_VERSION}")
ssl_context = ssl.create_default_context()
alpn_protocols = ['http/1.1']
ssl_context.set_alpn_protocols(alpn_protocols)
return ssl_context
except Exception as e:
print("Exception in ssl_alpn()")
raise e
connector_fqdn = ""
token = ""
file_name = "test.csv"
file_path = f"./{file_name}"
encoded_name = urllib.parse.quote(file_name)
# Use port 8883 for MQTT, could be changed to port 443 if configured from Murano product settings
host = f"{connector_fqdn}:8883"
endpoint = f"/onep:v1/content/{encoded_name}"
checksum = checksum(file_path)
checksum = base64.b64encode(checksum).decode('utf-8')
content_length = os.path.getsize(file_path)
headers = {
'X-Exosite-CIK': token,
'Content-Type': 'text/csv',
'Content-Length': str(content_length),
'Content-MD5': checksum
}
# This is the important option enabling use of HTTP over MQTT port
context = ssl_alpn()
conn = client.HTTPSConnection(host, context=context)
try:
with open(file_path, 'rb') as f:
data = f.read()
conn.request('PUT', endpoint, body=data, headers=headers, encode_chunked=False)
r = conn.getresponse()
print(f"STATUS: {str(r.status)}\nREASON: {r.reason}")
finally:
conn.close()
Misc ¶
Timestamp ¶
Get the current time according to the server.
Request
GET /timestamp HTTP/1.1
Host: <IOT_CONNECTOR_FQDN>
<blank_line>
Response
HTTP/1.1 200 OK
Date: <date>
Server: Murano
Connection: Keep-Alive
Content-Length: <length>
Content-Type: text/plain; charset=utf-8
<timestamp>
- See HTTP Responses for a full list of responses
Get Timestamp Example
$ curl -i https://<IOT_CONNECTOR_FQDN>/timestamp
HTTP/1.1 200 OK
Date: Fri, 05 May 2017 00:11:22 GMT
Server: Murano
Connection: Keep-Alive
Content-Length: 10
Content-Type: text/plain; charset=utf-8
1408088308