Skip to content

Time Series Storage Service

Summary

Write, record, and query device data in our Timeseries database optimized for IoT data.

TSDB (time series database) service provides a storage layer dedicated for metric data produced and evolving over time. For example, TSDB can be used to store periodic sensor data sent from your devices.

Data is associated with a timestamp and optional tags which can be used later for query purposes. Finally, the TSDB service provides a robust aggregation system enabling data-mining capabilities.

Data retention policy can be used to automatically delete data after a certain time period (days, weeks, months, years). When metric data is older than the specified retention period it will be permanently deleted during the next GC period. Because deletions happens periodically it is possible for queries to sometimes retrieve data older than the configured retention policy. The default retention policy applies to all metrics that do not have a per metric retention policy set.

Operations

Name Tag Summary
tsdb.addConfig() Storage add retention configuration of solution.
tsdb.deleteAll() Storage delete all data
tsdb.export() Storage Start an export job
tsdb.exportJobInfo() Storage Query the information of an export job, including status.
tsdb.exportJobList() Storage List export job records of a given solution in descending timestamp order
tsdb.getConfig() Storage Get retention configuration of solution.
tsdb.listMetrics() Storage list metrics of a given solution
tsdb.listTags() Storage list tags of a given solution
tsdb.multiWrite() Storage writes multiple datapoints into database for a specified product device of given solution
tsdb.query() Storage query the database
tsdb.recent() Storage get recent data
tsdb.write() Storage writes data into database for a specified product device of given solution

Events

Name Summary
exportJob Event message sent by TSDB

Configuration parameters

Name Type Description
default_retention ^(\d+(d|w|m|y)|infinity)$ Default retention time with unit. This applies to all metrics that do not have a per metric retention policy set. Default value is 3m.
Supported units: d (days), w (weeks), m (months), y (years)
e.g. 3d, 5w or infinity.
IMPORTANT this affect all the data store in this service.
Default: "3m"
metrics_retention object The retention time for metrics, property name should be the metric name.
metrics_retention.^[a-zA-Z0-9_]+$ ^(\d+(d|w|m|y)|infinity|default)$ Retention time for metric with unit.
To use default for following default value and infinity for keeping metric forever.
Supported units: d (days), w (weeks), m (months), y (years)
e.g. 3d, 5w, default or infinity.
recent_function_disable boolean Set to 'true' to disable the 'recent' functionality to optimize write requests. Using the 'recent' function will then return an error.

Operations

addConfig

Description

This operation is meant to set dynamic metric name retention which value won't be reflected to the service configuration. Will add or replace (if existing) metric retention configurations. For fixed metrics name set value from the service configuration instead.

Arguments

Name Type Description
default_retention ^(\d+(d|w|m|y)|infinity)$ Default retention time with unit. Default value is 3m.
Supported units: d (days), w (weeks), m (months), y (years)
e.g. 3d, 5w or infinity
Default: "3m"
metrics_retention object The retention time for metrics, property name should be the metric name.
metrics_retention.^[a-zA-Z0-9_]+$ ^(\d+(d|w|m|y)|infinity|default)$ Retention time for metric with unit.
To use default for following default value and infinity for keeping metric forever.
Supported units: d (days), w (weeks), m (months), y (years)
e.g. 3d, 5w, default or infinity

Responses

  • Returns nil for Config updated

  • Returns {object} when Error

    The response to the caller {object}

    Name Type Description
    error string Error Message in case of failure
    result object Result message

Example

local result = Tsdb.addConfig({
  default_retention = "3m",
  metrics_retention = {
    metric1 = "3d",
    metric2 = "10d"
  }
})
response.message = result

deleteAll

Description

delete all data of a given solution

Responses

  • Returns nil for All data deleted

  • Returns {object} when Error

    The response to the caller {object}

    Name Type Description
    error string Error Message in case of failure
    result object Result message

Example

-- Delete all data of a given solution
local out = Tsdb.deleteAll()
response.message = out

export

Description

Start an export job.
An event of TSDB export will be triggered once the job finished in any state.
You can define a 'tsdb' event handler in Lua for this 'exportJob' event in your solution.

Arguments

Name Type Description
query object Query arguments
query.fill integer, string Value to fill for time slots with no data points exist.
If the query without sampling, it only works in merge mode.
Supported fill types:
- "": fill the slot with empty string "".
- "null": fill the slot with JSON null.
- "previous": fill the slot with previous value.
- "none": fill the slot with "none" string.
- any integer: fill the slot with the specified integer value.
- "~s:CUSTOM_STRING": fill the slot with the specified "CUSTOM_STRING" string.
Default: none
query.mode string Indicate whether to merge or split the result of each metric.
Supported options: merge, split
Default: merge
query.tags object One or many tags.
Maximum number of tag pairs: 20
If tag value is string, it applies to “AND” operator with other tag pairs.
Following is example of operator (type=switch and area=US)
{
"tags": {
"type": "sensor",
"area": "US"
}
}
If the tag value is array, it applies to “OR” operator. All tag values with array type are in the same group of “OR” operator, even if the tag name is different. When OR operator appears in query, the response structure will be grouped by tags of OR operator.
Maximum number of "OR" tag values: 100
Following is example of operator (type=switch or type=sensor or area=US or area=TW), it counts the "OR" tag values as 4.
{
"tags": {
"type": ["switch", "sensor"],
"area": ["US", "TW"]
}
}
query.epoch string Change returned timestamp of data points to unix epoch format.
Supported units: u (microseconds), ms (milliseconds), s (seconds)
Optional, if not provided, timestamps are returned in RFC3339 UTC with microsecond precision. (Note: the time offset notation can be 'Z' or '+00:00')
query.limit integer Limit the number of data points to return per metric(default is 1000)
NOTE: When query datapoints with OR operator, the limit (default and maximum) will depends on number of OR tags have been provided. That is default = original default / number of OR tags
query.metrics [ string ] One or many metrics
query.end_time integer, string Exclusive UTC ending time range to query, also accept RFC3339 UTC string.
The end_time needs to bigger than or equal to start_time.
Supported units: u (microseconds), ms (milliseconds), s (seconds)
e.g. 1472547546000000u, 1472547546000ms, 1472547546s, 1472547546, 2016-08-30T08:59:06Z
Optional, if not provided, it will use current timestamp in microseconds from server side
query.order_by string Return results in ascending or descending time order.
Supported options: desc, asc
Default: "desc"
query.aggregate [ string ] One to many aggregation functions to apply.
Supported functions: avg, min, max, count, sum
String type value can only use count function
query.start_time integer, string Inclusive UTC starting time range to query, also accept RFC3339 UTC string.
Supported units: u (microseconds), ms (milliseconds), s (seconds)
e.g. 1472547546000000u, 1472547546000ms, 1472547546s, 1472547546, 2016-08-30T08:59:06Z, 2016-08-30T08:59:06+00:00
Optional, if not provided, it will be 7 days earlier than end_time.
query.relative_end integer, string A negative integer with time unit to indicate relative end time before now.
The relative_end time MUST be equal or bigger than the relative_start time.
Supported units: u (microseconds), ms (milliseconds), s (seconds), m (minutes), h (hours), d (days), w (weeks)
e.g. -3d (The data between 3 to 7 days ago when we use default relative_start time)
Optional, if not provided, the current server timestamp will be used.
query.sampling_size string The size of time slots used for downsampling. Must be a positive integer greater than zero.
Supported units: u (microseconds), ms (milliseconds), s (seconds), m (minutes), h (hours), d (days), w (weeks)
Optional, used together with fill arguments.
query.relative_start integer, string A negative integer with time unit to indicate relative start time before now.
Supported units: u (microseconds), ms (milliseconds), s (seconds), m (minutes), h (hours), d (days), w (weeks)
Default: -7d (last 7 days)
format object The data format rules. Property name should be the field name which are the metrics name, "timestamp" and "tags".
format.^[a-zA-Z0-9_]+$ [ object ] Functions to format the this field. the rules will be applied by its order in array.
format.^[a-zA-Z0-9_]+$[].label string Prepend the given string to field value. Metrics field only.
format.^[a-zA-Z0-9_]+$[].round integer Round the field value with given value. Metrics field only.
format.^[a-zA-Z0-9_]+$[].rename string Rename a field to the given value. Metrics field only.
format.^[a-zA-Z0-9_]+$[].discard boolean Remove the field when the value is true. The field parameter should be "tags" and only support "tags" field.
format.^[a-zA-Z0-9_]+$[].replace object Replace the field value which matching the pattern to the new value. Metrics field only.
format.^[a-zA-Z0-9_]+$[].replace.to string The replacement value. Using \{n} to specify capture group. {n} is number of group.
format.^[a-zA-Z0-9_]+$[].replace.match string String or regular expression.
format.^[a-zA-Z0-9_]+$[].datetime nil -
format.^[a-zA-Z0-9_]+$[].normalize [ string ] Normalize the given list of tag names, it will fill into different columns. The tags are not specified will be dropped.
The field parameter should be "tags" and only support "tags" field.
filename string File name of export CSV file. The ".csv" extension is not required inside the name. (Space is not allowed in filename)

Responses

  • Returns {object} when Job successfully started

    The information of job ID {object}

    Name Type Description
    job_id string Job ID
  • Returns {object} when Error

    The response to the caller {object}

    Name Type Description
    error string Error Message in case of failure
    result object Result message

Example

-- Query constraints for export
local metrics = {
  "temperature",
  "humidity",
  "switch",
  "host"
}
local tags = {
  region = "us",
  city = "minneapolis"
}
local query = {
  metrics = metrics,
  tags = tags
}
-- Some other possible dt_format examples: -- %Y-%m-%dT%H:%M:%S.%f%z (the default) -- %Y-%m-%d %H:%M:%S.%f (a human readable version with microseconds) -- %Y-%m-%d Winnipeg (you can append any string that works for your use case)
local format = {
  temperature = {
    {round=3},
    {rename="Temp"}
  },
  timestamp = {
    {datetime={offset=-5, dt_format="%Y-%m-%d %H:%M:%S UTC%Z America/Chicago"}}
  },
  tags = {
    {normalize={"city", "region"}}
  },
  host = {
    {replace={
      match="/(\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}):(\\d{1,5})/",
      to="Port: \\2 and IP: \\1"
    }}
  }
} -- Start a new export job
local job_id = Tsdb.export({
  query = query,
  filename = "export_mlps_20170321",
  format = format
})
response.message = job_id

exportJobInfo

Description

Query the information of an export job, including status.

Arguments

Name Type Description
job_id ^[a-zA-Z0-9-]+$ Job id

Responses

  • Returns {object} when Job info successfully returned

    The information for export job {object}

    Name Type Description
    error string Error message if job failed
    query object Query arguments
    state string State of the job (enqueued, expired, in-progress, completed or failed)
    format object The data format rules. Property name should be the field name which are the metrics name, "timestamp" and "tags".
    format.^[a-zA-Z0-9_]+$ [ object ] Functions to format the this field. the rules will be applied by its order in array.
    format.^[a-zA-Z0-9_]+$[].label string Prepend the given string to field value. Metrics field only.
    format.^[a-zA-Z0-9_]+$[].round integer Round the field value with given value. Metrics field only.
    format.^[a-zA-Z0-9_]+$[].rename string Rename a field to the given value. Metrics field only.
    format.^[a-zA-Z0-9_]+$[].discard boolean Remove the field when the value is true. The field parameter should be "tags" and only support "tags" field.
    format.^[a-zA-Z0-9_]+$[].replace object Replace the field value which matching the pattern to the new value. Metrics field only.
    format.^[a-zA-Z0-9_]+$[].replace.to string The replacement value. Using \{n} to specify capture group. {n} is number of group.
    format.^[a-zA-Z0-9_]+$[].replace.match string String or regular expression.
    format.^[a-zA-Z0-9_]+$[].datetime nil -
    format.^[a-zA-Z0-9_]+$[].normalize [ string ] Normalize the given list of tag names, it will fill into different columns. The tags are not specified will be dropped.
    The field parameter should be "tags" and only support "tags" field.
    job_id string Job ID
    length string The total length of export file in bytes
    filename string File name of the exported CSV file
    content_id string Content ID of the job to Content service
    context_id string Solution id
    start_time string Start time of the job
    update_time string Last updated time of the job
  • Returns {object} when Error

    The response to the caller {object}

    Name Type Description
    error string Error Message in case of failure
    result object Result message

Example

local job_info = Tsdb.exportJobInfo({
  job_id = "xxyyzz"
})
response.message = job_info

exportJobList

Description

List export job records of a given solution in descending timestamp order

Arguments

Name Type Description
limit integer Limit the number of results to return (default: 100, maximum allowed: 1000)

Responses

  • Returns {[ object ]} for List of export job

    Name Type Description
    state string State of the job (enqueued, expired, in-progress, completed or failed)
    job_id string Job ID
    start_time string Start time of the job
  • Returns {object} when Error

    The response to the caller {object}

    Name Type Description
    error string Error Message in case of failure
    result object Result message

Example

local job_list = Tsdb.exportJobList()
response.message = job_list

getConfig

Description

Get retention configuration of solution.

Responses

  • Returns {object} when The retention configuration successfully returned

    The configuration attributes for TSDB. {object}

    Name Type Description
    default_retention ^(\d+(d|w|m|y)|infinity)$ Default retention time with unit. Default value is 3m.
    Supported units: d (days), w (weeks), m (months), y (years)
    e.g. 3d, 5w or infinity
    Default: "3m"
    metrics_retention object The retention time for metrics, property name should be the metric name.
    metrics_retention.^[a-zA-Z0-9_]+$ ^(\d+(d|w|m|y)|infinity|default)$ Retention time for metric with unit.
    To use default for following default value and infinity for keeping metric forever.
    Supported units: d (days), w (weeks), m (months), y (years)
    e.g. 3d, 5w, default or infinity
  • Returns {object} when Error

    The response to the caller {object}

    Name Type Description
    error string Error Message in case of failure
    result object Result message

Example

local config = Tsdb.getConfig()
response.message = config

listMetrics

Description

list metrics of a given solution

Arguments

Name Type Description
limit integer Limit the number of rows to return.
Default: 1000
next string Optional, the cursor to get next page if still having more data

Responses

  • Returns {object} when Metrics information retrieved

    Metric list {object}

    Name Type Description
    next string Cursor for getting next page
    total integer Total number of items
    metrics [ string ] List of metrics
  • Returns {object} when Error

    The response to the caller {object}

    Name Type Description
    error string Error Message in case of failure
    result object Result message

Example

-- Get a list of created metrics for a given solution
local out = Tsdb.listMetrics({limit = 10})
response.message = out

-- Use next cursor to fetch next page if found in the result of previous query
local out = Tsdb.listMetrics({limit = 10, next = out.next})
response.message = out

listTags

Description

list tags of a given solution

Arguments

Name Type Description
limit integer Limit the number of rows to return.
Default: 1000
next string Optional, the cursor to get next page if still having more data

Responses

  • Returns {object} when Tags information retrieved

    Tag list {object}

    Name Type Description
    next string Cursor for getting next page
    tags object Map of tags
    total integer Total number of items
  • Returns {object} when Error

    The response to the caller {object}

    Name Type Description
    error string Error Message in case of failure
    result object Result message

Example

-- Get a list of created tags for a given solution
local out = Tsdb.listTags({limit = 10})
response.message = out

-- Use next cursor to fetch next page if found in the result of previous query
local out = Tsdb.listTags({limit = 10, next = out.next})
response.message = out

multiWrite

Description

Write data points to one or many metrics with an optional set of tags and a timestamp down to microsecond precision.

Note that if multiple data points are written with exactly the same timestamp, only the last one will be kept and it overwrites the others.

Each metric value has a limited size which depends on the number of tags. (Number of tags + 1) multiplies the size of metric value can't over 480KB. A write request will be rejected without partial writes if exceeding the limit. If any of data points are invalid, request will be rejected without partial writes.

To prevent from a synchronized service call take too long time to response, there are some limitations. Total number of data entries in multiple write at most 2,000 (Reference to the limit of single write). Number of data entries per datapoint: "Number of metrics * (Number of tag pairs) + 1"

If succeeds, it turns a list string of write timestamp in microseconds.

To improve write performance, the recent functionality can also be disabled by setting the recent_function_disable option to true.

Arguments

Name Type Description
recent_function_disable boolean Disable the recent functionality or not.
return_ts boolean Whether to return write timestamp in the response
datapoints [ object ] List of data points
datapoints[].ts integer, string Unix timestamp in microseconds used as the write time for given data point.
Supported units: u (microseconds), ms (milliseconds), s (seconds)
e.g. 1472547546000000u, 1472547546000ms, 1472547546s, 1472547546
Optional, if not provided, it will use the received time in microseconds from server side
datapoints[].tags object Pairs of tag and its tag value (only text supported).
Maximum number of tags in a single write: 20
datapoints[].metrics object Pairs of metric name and its value.
Maximum number of metrics in a single write: 100

Responses

  • Returns {[ object ]} when Data successfully inserted

    List of result of each data point. {object}

    Name Type Description
    write_timestamp string The timestamp of data point written to TSDB (in microseconds)
  • Returns nil for Data successfully inserted

  • Returns {object} when Error

    The response to the caller {object}

    Name Type Description
    error string Error Message in case of failure
    result object Result message

Example

-- Write multiple datapoints of metrics with tags
-- If timestamp is not provided, it will use the received time in microseconds from server side
local metrics1 = {
  temperature = 37.2,
  humidity = 73,
  switch = "on"
}
local metrics2 = {
  temperature = 31.2,
  humidity = 55,
  switch = "off"
}
local tags1 = {
  pid = "pzomp8vn4twklnmi",
  identity = "000001",
  region = "us",
  city = "minneapolis"
}
local tags2 = {
  pid = "lvwpoj19hp7k0000",
  identity = "000002",
  region = "tw",
  city = "taipei"
}
local out = Tsdb.multiWrite({
  datapoints = {
    {
      metrics = metrics1,
      tags = tags1
    },
    {
      metrics = metrics2,
      tags = tags2
    }
  },
  return_ts = true
})
response.message = out

query

Description

Query data points by using any metrics and tags. Support absolute (start_time, end_time) or relative (relative_start, relative_end) time parameters. The end time MUST be bigger or equal to the start time (end_time >= start_time or relative_end >= relative_start).
The first element in returned data point array is always the timestamp.
You can use fill argument to indicate the imputation of missing values.
The metric names of columns property in the response will always be in the order specified in the query, except for the timestamp column which is always the first one.

If no time constraints are specified, it will return recent data points up to the maximum limit in recent one week.

Note that only unique timestamped data will be returned, which means if multiple data points were written with exactly the same timestamp in the response, only the last one will be kept. The logic OR operator is only supported in basic query and cannot be used in down-sampling or aggregation queries.

Arguments

Name Type Description
fill integer, string Value to fill for time slots with no data points exist.
If the query without sampling, it only works in merge mode.
Supported fill types:
- "": fill the slot with empty string "".
- "null": fill the slot with JSON null.
- "previous": fill the slot with previous value.
- "none": fill the slot with "none" string.
- any integer: fill the slot with the specified integer value.
- "~s:CUSTOM_STRING": fill the slot with the specified "CUSTOM_STRING" string.
Default: none
mode string Indicate whether to merge or split the result of each metric.
Supported options: merge, split
Default: merge
tags object One or many tags.
Maximum number of tag pairs: 20
If tag value is string, it applies to “AND” operator with other tag pairs.
Following is example of operator (type=switch and area=US)
{
"tags": {
"type": "sensor",
"area": "US"
}
}
If the tag value is array, it applies to “OR” operator. All tag values with array type are in the same group of “OR” operator, even if the tag name is different. When OR operator appears in query, the response structure will be grouped by tags of OR operator.
Maximum number of "OR" tag values: 100
Following is example of operator (type=switch or type=sensor or area=US or area=TW), it counts the "OR" tag values as 4.
{
"tags": {
"type": ["switch", "sensor"],
"area": ["US", "TW"]
}
}
epoch string Change returned timestamp of data points to unix epoch format.
Supported units: u (microseconds), ms (milliseconds), s (seconds)
Optional, if not provided, timestamps are returned in RFC3339 UTC with microsecond precision. (Note: the time offset notation can be 'Z' or '+00:00')
limit integer Limit the number of data points to return per metric(default is 1000)
NOTE: When query datapoints with OR operator, the limit (default and maximum) will depends on number of OR tags have been provided. That is default = original default / number of OR tags
metrics [ string ] One or many metrics
end_time integer, string Exclusive UTC ending time range to query, also accept RFC3339 UTC string.
The end_time needs to bigger than or equal to start_time.
Supported units: u (microseconds), ms (milliseconds), s (seconds)
e.g. 1472547546000000u, 1472547546000ms, 1472547546s, 1472547546, 2016-08-30T08:59:06Z
Optional, if not provided, it will use current timestamp in microseconds from server side
order_by string Return results in ascending or descending time order.
Supported options: desc, asc
Default: "desc"
aggregate [ string ] One to many aggregation functions to apply.
Supported functions: avg, min, max, count, sum
String type value can only use count function
start_time integer, string Inclusive UTC starting time range to query, also accept RFC3339 UTC string.
Supported units: u (microseconds), ms (milliseconds), s (seconds)
e.g. 1472547546000000u, 1472547546000ms, 1472547546s, 1472547546, 2016-08-30T08:59:06Z, 2016-08-30T08:59:06+00:00
Optional, if not provided, it will be 7 days earlier than end_time.
relative_end integer, string A negative integer with time unit to indicate relative end time before now.
The relative_end time MUST be equal or bigger than the relative_start time.
Supported units: u (microseconds), ms (milliseconds), s (seconds), m (minutes), h (hours), d (days), w (weeks)
e.g. -3d (The data between 3 to 7 days ago when we use default relative_start time)
Optional, if not provided, the current server timestamp will be used.
sampling_size string The size of time slots used for downsampling. Must be a positive integer greater than zero.
Supported units: u (microseconds), ms (milliseconds), s (seconds), m (minutes), h (hours), d (days), w (weeks)
Optional, used together with fill arguments.
relative_start integer, string A negative integer with time unit to indicate relative start time before now.
Supported units: u (microseconds), ms (milliseconds), s (seconds), m (minutes), h (hours), d (days), w (weeks)
Default: -7d (last 7 days)

Responses

  • Returns {object} when Error

    The response to the caller {object}

    Name Type Description
    error string Error Message in case of failure
    result object Result message
  • Returns {object} when Query results retrieved in merge mode.

    Response data in merge mode. {object}

    Name Type Description
    tags object The “AND” operator tags specified in query arguments.
    values [ [ number, string, object ] ] The data points list.
    columns [ string ] The column names for mapping the column in values property.
    metrics [ string ] Metrics which are specified in query arguments.
  • Returns {object} when Query results retrieved in split mode.

    Response data in split mode. {object}

    Name Type Description
    tags object The “AND” operator tags specified in query arguments.
    values object The data points list for each metric, the property name is the metric name specified in query arguments.
    values.^[a-zA-Z0-9_]+$ [ [ number, string, object ] ] The data points list.
    columns [ string ] Its always empty array in this mode.
    metrics [ string ] Metrics which are specified in query arguments.
  • Returns {object} when Query results retrieved in merge mode with OR operation.

    Response data in merge mode with OR operation. {object}

    Name Type Description
    tags object The “AND” operator tags specified in query arguments.
    values object The data points list of specific tag name, the property name is tag name.
    values.^[a-zA-Z0-9_]+$ object The data points list of specific tag name and tag value, the property name is tag value.
    values.[a-zA-Z0-9_]+$.[a-zA-Z0-9_]+$ [ [ number, string, object ] ] The data points list.
    columns [ string ] The column names for mapping the column in values property.
    metrics [ string ] Metrics which are specified in query arguments.
  • Returns {object} when Query results retrieved in split mode with OR operation.

    Response data in split mode with OR operation. {object}

    Name Type Description
    tags object The “AND” operator tags specified in query arguments.
    values object The data points list of specific tag name, the property name is tag name.
    values.^[a-zA-Z0-9_]+$ object The data points list of specific tag name and tag value, the property name is tag value.
    values.[a-zA-Z0-9_]+$.[a-zA-Z0-9_]+$ object The data points list for each metric, the property name is the metric name specified in query arguments.
    values.[a-zA-Z0-9_]+$.[a-zA-Z0-9_]+$.^[a-zA-Z0-9_]+$ [ [ number, string, object ] ] The data points list.
    columns [ string ] Its always empty array in this mode.
    metrics [ string ] Metrics which are specified in query arguments.
  • Returns {object} when Query results retrieved in aggregation mode.

    Response data in split mode or aggregate funs specified in query arguments. {object}

    Name Type Description
    tags object The “AND” operator tags specified in query arguments.
    values object The data points list for each metric, the property name is the metric name specified in query arguments.
    values.^[a-zA-Z0-9_]+$ object The aggregation of data points.
    values.^[a-zA-Z0-9_]+$.avg number The average of metric values.
    values.^[a-zA-Z0-9_]+$.max number The maximum of metric values.
    values.^[a-zA-Z0-9_]+$.min number The minimum of metric values.
    values.^[a-zA-Z0-9_]+$.sum number The sum of metric values.
    values.^[a-zA-Z0-9_]+$.count number The count of metric values.
    columns [ string ] Its always empty array in this mode.
    metrics [ string ] Metrics which are specified in query arguments.

Example

-- Query by Absolute Time Constraint --
-- Get temperature and humidity data points between 2016-08-01 (inclusive) and 2016-09-01 (exclusive) from devices in Minneapolis city and US region
local metrics = {
  "temperature",
  "switch"
}
local tags = {
  region = "us",
  city = "minneapolis"
}
local out = Tsdb.query({
  metrics = metrics,
  tags = tags,
  start_time = "2016-08-01T00:00:00Z",
  end_time = "2016-09-01T00:00:00Z",
  fill = "null",
  limit = 50
})
response.message = out

-- Query by Relative Time Constraint --
-- Get temperature data points in recent 3 hours from devices in Taipei city and Asia region (with timestamp in milliseconds format)
local metrics = {"temperature"}
local tags = {
  region = "asia",
  city = "taipei"
}
local out = Tsdb.query({
  metrics = metrics,
  tags = tags,
  relative_start = "-3h",
  epoch = "ms",
  fill = "null",
  limit = 50
})
response.message = out
-- Get temperature data points between 3 to 10 days ago from devices in Taipei city and Asia region (with timestamp in milliseconds format)
local metrics = {"temperature"}
local tags = {
  region = "asia",
  city = "taipei"
}
local out = Tsdb.query({
  metrics = metrics,
  tags = tags,
  relative_start = "-10d",
  relative_end = "-3d",
  epoch = "ms",
  fill = "null",
  limit = 50
})
response.message = out

-- Query without Time Constraint --
-- Get most recent 5 temperature and humidity data points from devices in Minneapolis city and US region
local metrics = {
  "temperature",
  "humidity"
}
local tags = {
  region = "us",
  city = "minneapolis"
}
local out = Tsdb.query({
  metrics = metrics,
  tags = tags,
  limit = 5
})
response.message = out

-- Query by Downsampling --
-- Get humidity data points in recent two days from devices in Taipei city, downsampled by 4-hours time slots
local metrics = {
  "humidity"
}
local tags = {
  city = "taipei"
}
local out = Tsdb.query({
  metrics = metrics,
  tags = tags,
  relative_start = "-2d",
  sampling_size = "4h",
  fill = "none",
  epoch = "ms"
})
response.message = out

-- Aggregation by Downsampling --
-- Get average and count of tire pressure data between 2016-08-15 (inclusive) and 2016-09-01 (exclusive) from devices in Minneapolis city, downsampled by 30-minutes time slots
local metrics = {
  "tire_pressure"
}
local tags = {
  city = "minneapolis"
}
local aggregate = {"avg", "count"}
local out = Tsdb.query({
  metrics = metrics,
  tags = tags,
  start_time = "2016-08-01T00:00:00Z",
  end_time = "2016-09-01T00:00:00Z",
  aggregate = aggregate,
  sampling_size = "30m",
  fill = "none"
})
response.message = out
-- Query by Fill in Custom String --
-- To fill in the empty time slots with "Empty" string
local metrics = {
  "temperature",
  "switch"
}
local tags = {
  region = "us",
  city = "minneapolis"
}
local out = Tsdb.query({
  metrics = metrics,
  tags = tags,
  fill = "~s:Empty",
})
response.message = out
-- Query by OR tags operator --
-- Get temperature data points which belongs to (sn = dev1 or sn = dev2) and dev = pump
local metrics = {
  "temperature"
} local tags = {
  dev = "pump",
  sn = {"dev1", "dev2"}
}
local out = Tsdb.query({
  metrics = metrics,
  tags = tags,
  start_time = "2016-08-01T00:00:00Z",
  end_time = "2016-09-01T00:00:00Z",
  limit = 50
})
response.message = out

recent

Description

Get the most recent data point of a particular set of metrics and tag values.

If you want to use advanced metric query, you need to specify an inner Lua table as an element
inside the Lua table of metrics parameter, i.e. metrics = {"m1","m2",{"m3","m4"}}
where {"m3","m4"} is an advanced metric query that query the most recent data
of m3 and also report the value of m4 that written together with m3.

To improve write request performance, this functionality can be disabled by setting the recent_function_disable option to true. If disabled the using the 'recent' query will return an error.

Arguments

Name Type Description
recent_function_disable boolean Disable the recent functionality or not.
metrics [ object ] One or many metrics
tag_name string Tag name
tag_values [ integer, string ] One or many tag values

Responses

  • Returns nil for Operation successfully returned

  • Returns {object} when Error

    The response to the caller {object}

    Name Type Description
    error string Error Message in case of failure
    result object Result message

Example

-- Get latest data of metric vibration and humidity from devices with tag sn=123 or sn=456
local out = Tsdb.recent({
  metrics = {"vibration","humidity"},
  tag_name = "sn",
  tag_values = {"123", "456"}
})
response.message = out

-- Get latest data of metric warning and critical from devices with tag sn=123 or sn=456
-- Together with the corresponding text for warning and critical metrics
local out = Tsdb.recent({
  metrics = { {"warning", "text"}, {"critical", "text"} },
  tag_name = "sn",
  tag_values = {"123", "456"}
})
response.message = out

write

Description

Write data point to one or many metrics with an optional set of tags and a timestamp down to microsecond precision.

Note that if multiple data points are written with exactly the same timestamp, only the last one will be kept and it overwrites the others.

Each metric value has a limited size which depends on the number of tags. (Number of tags + 1) multiplies the size of metric value can't over 480KB. A write request will be rejected without partial writes if exceeding the limit.

If succeeds, it returns a json of write timestamp in microseconds.

Important note on Tag usage: Tags applies a multiplication factor on the metric (eg. a metric with 4 tags -> 4x data write) and will have an impact on both performance and storage usage.

To improve write performance, the recent functionality can also be disabled by setting the recent_function_disable option to true.

Arguments

Name Type Description
recent_function_disable boolean Disable the recent functionality or not.
ts integer, string Unix timestamp in microseconds used as the write time for given data point.
Supported units: u (microseconds), ms (milliseconds), s (seconds)
e.g. 1472547546000000u, 1472547546000ms, 1472547546s, 1472547546
If the unit is not provided, it will be chosen by the number of digits to avoid the user from unintentionally using unusual UTC timestamps.
For seconds precision, the length of UTC timestamp, it is always 10 digits in between 2001/09/09 (9 digits) and 2286/11/20 (11 digits), which we assume that the "real data" be written is in the range of 10 digits.
Similarly, for milliseconds (seconds times 10^3) and microseconds (seconds times 10^6), the reasonable number of digits are 13 and 16 respectively.
For example, the unit of 1472547546 would be second and 14725475460 is invalid.
Valid time range: 1,000,000,000,000,000(us) to 9,999,999,999,999,999(us) unix timestamp.
Optional, if not provided, it will use the received time in microseconds from server side
tags object Pairs of tag and its tag value (only text supported).
Maximum size of tag name and tag value: 1KB.
Maximum number of tags in a single write: 20
metrics object Pairs of metric name and its value.
Maximum size of metric name: 1KB.
Maximum number of metrics in a single write: 100
return_ts boolean Whether to return write timestamp in the response

Responses

  • Returns {object} when Data successfully inserted

    The response of write {object}

    Name Type Description
    write_timestamp string The timestamp of data point written to TSDB (in microseconds)
  • Returns nil for Data successfully inserted

  • Returns {object} when Error

    The response to the caller {object}

    Name Type Description
    error string Error Message in case of failure
    result object Result message

Example

-- Write data point of metrics with tags
-- If timestamp is not provided, it will use the received time in microseconds from server side
local metrics = {
  temperature = 37.2,
  humidity = 73,
  switch = "on",
  host = "8.168.1.24:443"
}
local tags = {
  pid = "pzomp8vn4twklnmi",
  identity = "000001",
  region = "us",
  city = "minneapolis"
}
local out = Tsdb.write({
  metrics = metrics,
  tags = tags
})
response.message = out

-- Write data points of metrics with tags and timestamp
local metrics = {
  temperature = 37.2,
  humidity = 73
}
local tags = {
  identity = "000002"
}
local out = Tsdb.write({
  metrics = metrics,
  tags = tags,
  ts = "1476243965s"
})
response.message = out

Events

exportJob

Description

An event message containing the export task result.

Arguments

Name Type Description
error string Error message if job failed
query object Query arguments
state string State of the job (enqueued, expired, in-progress, completed or failed)
format object The data format rules. Property name should be the field name which are the metrics name, "timestamp" and "tags".
format.^[a-zA-Z0-9_]+$ [ object ] Functions to format the this field. the rules will be applied by its order in array.
format.^[a-zA-Z0-9_]+$[].label string Prepend the given string to field value. Metrics field only.
format.^[a-zA-Z0-9_]+$[].round integer Round the field value with given value. Metrics field only.
format.^[a-zA-Z0-9_]+$[].rename string Rename a field to the given value. Metrics field only.
format.^[a-zA-Z0-9_]+$[].discard boolean Remove the field when the value is true. The field parameter should be "tags" and only support "tags" field.
format.^[a-zA-Z0-9_]+$[].replace object Replace the field value which matching the pattern to the new value. Metrics field only.
format.^[a-zA-Z0-9_]+$[].replace.to string The replacement value. Using \{n} to specify capture group. {n} is number of group.
format.^[a-zA-Z0-9_]+$[].replace.match string String or regular expression.
format.^[a-zA-Z0-9_]+$[].datetime nil -
format.^[a-zA-Z0-9_]+$[].normalize [ string ] Normalize the given list of tag names, it will fill into different columns. The tags are not specified will be dropped.
The field parameter should be "tags" and only support "tags" field.
job_id string Job ID
length string The total length of export file in bytes
filename string File name of the exported CSV file
content_id string Content ID of the job to Content service
context_id string Solution id
start_time string Start time of the job
update_time string Last updated time of the job

Example

function handle_tsdb_exportJob (job)

 -- Your logic comes here 

end