Transforms¶

Overview¶

Signal Transformations allow for running real-time streaming algorithms and logic on signals for further processing of the data such as running math on, complex functions, and combining with other signals. The output is always one or more unique signals for the asset that can be used like any other signal.

Examples of common use cases for Transforms:

Scores / Metrics: Calculating the health of equipment based on one or more streams of signal data.
Combining: Combining signals to calculate another piece of information.
Aggregation: Aggregate data such as calculating sum, count, min, max, average, and standard deviation.
Filters / Sampling: Filtering data (e.g. Band Gap, Errors) or sampling of data to lower amount of data points stored over time.
Accumulation: Accumulate or Sum data (e.g. accumulated over a time period, convert data to a estimated volume or mass from flow)
Conversions: Signal data can be mapped or mathematically converted to an appropriate (Gain, Mapping, Joins) Note: Unit conversions are natively supported in the ExoSense application
Insights & Analytics: Combining data with third-party external data sets and algorithms, integration with analytics and machine learning. These leverage External Insights functionality.

Transformations convert a signal to a new signal with some math applied, a join with other signals, or with more complex processing. The output of a transformation is a unique signal that can be used as a part of the asset for rules, dashboards, etc.

---
config:
theme: neutral
---
graph LR
Signal@{ shape: das, label: "Signal" }
OutSignal@{ shape: das, label: "Output Signal" }
d0@{ shape: framed-circle, label: "Data Point" }
d1@{ shape: framed-circle, label: "Data Point" }
d2@{ shape: framed-circle, label: "Data Point" }
d3@{ shape: sm-circ, label: "Data Point" }
Transform[/Transform/];
style Transform stroke-width:3px
DPC@{ shape: brace-r, label: "New Incoming Signal Values" }
EVENTC@{ shape: brace-l, label: "The output signal's new value" }
DPC ~~~ d0
d0 e1@-.- d1 e2@-.- d2 e3@-.- Signal e4@-.- d3 e5@--> Transform
%%d3 eTS@-.-o ts
state_change@{ shape: sm-circ }
style state_change fill:#2C9DB6
Transform e6@-.-  OutSignal e7@-.- | Calculated<br>Value | state_change
state_change ~~~ EVENTC
e1@{ animate: true }
e2@{ animate: true }
e3@{ animate: true }
e4@{ animate: true }
e5@{ animate: true }
e6@{ animate: true }
e7@{ animate: true }

Multiple input and output signals are supported by transform functions.

---
config:
theme: neutral
---
graph LR
Signal@{ shape: das, label: "Signal 1" }
OutSignal@{ shape: das, label: "Output 1 Signal" }
OutSignalTwo@{ shape: das, label: "Output 2 Signal" }

SignalTwo@{ shape: das, label: "Signal 2" }
d0@{ shape: framed-circle, label: "Data Point" }
d1@{ shape: framed-circle, label: "Data Point" }
d2@{ shape: framed-circle, label: "Data Point" }
d3@{ shape: sm-circ, label: "Data Point" }
Transform[/Transform/];
style Transform stroke-width:3px
DPC@{ shape: brace-r, label: "New Incoming Signal Values" }
EVENTC@{ shape: brace-l, label: "Each output signal may or may not be updated depending on the Logic" }
DPC ~~~ d0
d0 e1@-.- d1 e2@-.- d2 e3@-.- Signal e4@-.- d3 e5@--> Transform
SignalTwo -..-o | Most Recent<br>Value | Transform
state_change@{ shape: sm-circ }
style state_change fill:#2C9DB6
state_change2@{ shape: sm-circ }
style state_change2 fill:#2C9DB6
Transform e6@-.-  OutSignal e7@-.- | Calculated<br>Value | state_change
Transform e8@-.-  OutSignalTwo e9@-.- | Calculated<br>Value | state_change2
state_change ~~~ EVENTC
e1@{ animate: true }
e2@{ animate: true }
e3@{ animate: true }
e4@{ animate: true }
e5@{ animate: true }
e6@{ animate: true }
e7@{ animate: true }
e8@{ animate: true }
e9@{ animate: true }

Transform functions are event driven, running on each new value from each input signal. For use cases where a transform has multiple signal inputs, it's important to remember that the function will run with the new value of one of the signals and with the most recent value(s) of the other signal(s).

ExoSense built-in transforms

ExoSense includes some built-in functions available for a number of common data pipeline use cases.

Function	Type	Description
Linear Gain	Standard	Linear gain + offset function.
Linear Map	Standard	Map to new value based on a linear scale.
Join	Standard	Join multiple signals with basic operations.
Continuous Accumulation	Standard	Accumulate/Decrement signal values
Infinite Impulse Response (IIR)	Standard	Infinite Impulse Response
Negative Temperature Coefficient (NTC)	Standard	Calculate Temperature from NTC sensor
Band Gap Filter	Standard	Filter in / out of a band gap
Total Mass from Flow Mass	Flow	Get Total Mass from Flow Mass over time
Total Volume from Flow Volume per Minute	Flow	Get Volume from Flow Volume over time
Total Volume from Flow Volume per Second	Flow	Get Volume from Flow Volume over time
Sum/Count/Min/Max/Avg in Time Interval	Windowed	Time Interval based Aggregation Operations
Standard Deviation in Time Interval	Windowed	Time Interval based Standard Deviation
Sample Standard Deviation in Time Interval	Windowed	Time Interval based Sample Standard Deviation
Rolling Aggregation by Count	Windowed	Rolling aggregation by count

Using Transforms¶

In the Modify Asset Config interface for any given Asset, transforms can be added to the existing signals. Once added, one or more new signals are created.

The output Signals will be listed in Modify Asset Configuration under 'Calculated Signals' and are available for adding rules to, further transformations or analytics, reporting, and dashboard use.

Custom Transform Functions¶

The off-the-shelf standard transformations provided with ExoSense may not cover all use cases. Custom transforms can be created using the ExoSense Insight functionality.

Learn More: Insight Functions

Inline Insight Functions

This feature allows solution administrators to create custom transform functions, defining function inputs, outputs, constants, and logic within the ExoSense application and does not require any additional web services or integrations.

External Insight Functions

The Exosite IoT Marketplace contains additional Insight modules that can be added to ExoSense by administrators. Custom transformations can be built for transforming of data or analytics. Once added or created, these show up in the Modify Asset Config 'Add Transform' interface. Creating external insight integrations is an advanced topic.

External Insights can leverage webservices external to the platform to handle integrations with third-party data, algorithms, machine learning, and other services. Deploying external Insight Modules for External Transforms is an advanced developer topic. Please contact Exosite's team to discuss your application needs.

---
config:
theme: neutral
---
graph LR
Signal@{ shape: das, label: "Signal" }
%%SignalTwo@{ shape: das, label: "Signal 2" }
%%SignalThree@{ shape: das, label: "Signal 3" }
OutSignal@{ shape: das, label: "Output Signal" }

d0@{ shape: framed-circle, label: "Data Point" }
d1@{ shape: framed-circle, label: "Data Point" }
d2@{ shape: framed-circle, label: "Data Point" }
d3@{ shape: sm-circ, label: "Data Point" }

Transform[/"External Transform"/]
style Transform stroke-width:3px
External[External Service]

d0 e1@-.- d1 e2@-.- d2 e3@-.- Signal e4@-.- d3 e5@-->  Transform
%%SignalTwo -..-o | Most Recent<br>Value | Transform
%%SignalThree -..-o | Most Recent<br>Value | Transform
state_change@{ shape: sm-circ }
style state_change fill:#2C9DB6
Transform e8@-..->External e9@-..-> Transform
Transform e6@-.->  OutSignal e7@-.- | Calculated<br>Value | state_change
EVENTC@{ shape: brace-l, label: "Value from External Insight Service" }
state_change ~~~ EVENTC
EXTC@{ shape: brace-l, label: "Algorithms, Machine Learning, Data Sets, Third-party Data" }
External ~~~ EXTC
e1@{ animate: true }
e2@{ animate: true }
e3@{ animate: true }
e4@{ animate: true }
e5@{ animate: true }
e6@{ animate: true }
e7@{ animate: true }
e8@{ animate: true }
e9@{ animate: true }

Built-in Transforms¶

Standard Transform Details¶

Linear Gain¶

Allows for applying a scaling factor and offset for a signal using the function: result = gain * input + offset.

Use Cases:

Use Linear Scale for applying calibration factors to signals.

Configuration

I/O	Details
Inputs	1 Numeric signal
Outputs	1 Numeric signal (Type / Unit can be configured)

Parameters:

Parameter	Format	Required	Description
Gain	Number	✔︎	Multiplier of the signal value before the offset is applied. Default to 1. Can be negative.
Offset	Number	✔︎	Added to the value after the gain is applied. Default to 0. Can be negative. Can be negative.

Linear Map¶

Map a signal to new values, based on a linear scale. Enter the original signal's minimum and maximum, and the transformed signal's minimum and maximum. Computes the Linear Map from In Minimum and In Maximum to Out Minimum and Out Maximum (result = (x - inMin) * ∆Out / ∆In + outMin)

Use Cases:

Use Linear Mapping to transform a hardwired analog input 4-20mA signal to a 0-100% range signal.

Configuration

I/O	Details
Inputs	1 Numeric signal
Outputs	1 Numeric signal (Type / Unit can be configured)

Parameters:

Parameter	Format	Required	Description
In Minimum	Number	✔︎	Minimum input value to use in the mapping function.
In Maximum	Number	✔︎	Maximum input value to use in the mapping function.
Out Minimum	Number	✔︎	Minimum output value to use in the mapping function.
Out Maximum	Number	✔︎	Maximum outputs value to use in the mapping function.

Join¶

Join multiple signals with basic math operations.

addition
subtraction
multiply
divide
modulo

Evaluation Sequence

Join operations are evaluated top-to-bottom, rather than per standard order of operations.

Triggers

The join transform is triggered for each input signal value, meaning that if both signals have data written to them through a device at the same time, the transform will run twice. If your Join has two signals, A and B, and both values are written into the system at the same time, the function will for example run once for input A (using a previous B value) and once for input B (with the latest A value).

Use Cases:

Example: Use Join to add energy meters together for a combined energy usage signal.

Configuration

I/O	Details
Inputs	2 to 5 Numeric signals
Outputs	1 Numeric signal (Type / Unit can be configured)

Parameters:

Parameter	Format	Required	Description
First Op	Number	✔︎	Math operations to be applied between the First input and Second input.
Second Op	Number		Math operations to be applied between the Second input and Third input.
Third Op	Number		Math operations to be applied between the Third input and Fourth input.
Fourth Op	Number		Math operations to be applied between the Fourth input and Fifth input.
Gain	Number	✔︎	Default is 1. Multiplier of the signal value before the offset is applied. Default to 1. Can be negative.
Offset	Number	✔︎	Default is 0. Added to the value after the gain is applied. Default to 0. Can be negative. Can be negative.

Continuous Accumulation¶

This function will Sum (Increment) or Subtract (Decrement) incoming values with an optional starting value (default 0).

Use Cases:

Continuous accumulation of a sensor value over the lifetime

Configuration

I/O	Details
Inputs	1 Numeric signal
Outputs	1 Numeric signal (Type / Unit can be configured)

Parameters:

Parameter	Format	Required	Description
Value to start from	Number	✔︎	Default 0. The starting number for increment/decrementing from

Infinite Impulse Response (IIR)¶

This function will combine incoming values by an IIR [https://en.wikipedia.org/wiki/Infinite_impulse_response].

Configuration

I/O	Details
Inputs	1 Numeric signal
Outputs	1 Numeric signal (Type / Unit can be configured)

Parameters:

Parameter	Format	Required	Description
Weighting Value	Number	✔︎	Decimal value between 0 and 1. A value between 0 and 1 excluding to weight prior values

Negative Temperature Coefficient (NTC)¶

Computes Temperature from an analog voltage value using NTC provided Beta and Resistance at 25°C parameters. [https://en.wikipedia.org/wiki/Temperature_coefficient]

Configuration

I/O	Details
Inputs	1 Numeric signal (assumes an analog voltage)
Outputs	1 Temperature signal (℃)

Parameters:

Parameter	Format	Required	Description
Beta	Number	✔︎	Beta value from sensor datasheet
Resistance at 25℃	Number	✔︎	Resistance at 25℃ value from sensor datasheet

Band Gap Filter¶

This function will sort values that are inside and outside of a range into two output signals.

Configuration

I/O	Details
Inputs	1 Numeric signal
Outputs	2 signals using the original input signal type/unit

Parameters:

Parameter	Format	Required	Description
Lower Limit	Number	✔︎	Exclusive lower limit for band gap
Upper Limit	Number	✔︎	Exclusive upper limit for band gap

Flow Transformations¶

Total Mass from Flow Mass¶

Derives the total mass from incoming flow mass rate.

Configuration

I/O	Details
Inputs	1 signal of Flow Mass type
Outputs	1 signal of Mass type

Total Volume from Flow Volume per Minute¶

Derives the total volume from incoming flow volume per minute.

Configuration

I/O	Details
Inputs	1 signal of Flow Volume type, units of per minute
Outputs	1 signal of Volume type

Total Volume from Flow Volume per Second¶

Derives the total volume from incoming flow volume per second.

Configuration

I/O	Details
Inputs	1 signal of Flow Volume type, units of per second
Outputs	1 signal of Volume type

Windowed Transforms¶

Windowed functions operate with a time window concept and/or a count as a part of the algorithm but use online algorithm concepts rather than rely on requesting history. As with all Transform streaming functions, these are called and operate real-time for every input signal value. Each input value will trigger a new output value.

Sum, Count, Min, Max, Avg - in Time Interval¶

Operational functions supported: Sum, Count, Minimum, Maximum, Average

These functions operate to continuously calculate a output value using an aggregation operation (sum, count, average, maximum, minimum) with the latest signal value within time intervals (e.g. every hour). These aggregation operation functions use a concept of an online algorithm.

Learn more about Online Algorithms: https://en.wikipedia.org/wiki/Online_algorithm
Intervals are based on the UTC time and allows specifying an offset.
Note: Timezones and seasonal time changes are unknown to these operation functions.

Configuration

I/O	Details
Input	Any Numeric signal may be used with this transform function.
Output	The calculated value based on the aggregation operation type for the given input signal. The Type / Unit can be configured depending on the operation being performed.

Parameters:

Parameter	Format	Required	Description
Interval Size	Enum (a time interval)	✔︎	Selectable Time Interval for operation. (10 minutes, 30 minutes, 1 hour, 24 hours )
Interval Alignment	Enum (an offset time period)	✔︎	Selectable Offset from the hourly UTC time for the Interval. (0, 15m, 30m, 45m, 1h, 2h, ..., 23h)

Signal History is Not Required to reduce storage usage

To save on signal storage history, the output signal's history property may be disabled for any outputs that do not need history to be viewed or reported on. History of any of the output signals is not required for this function.

Standard Deviation in Time Interval¶

Operates on a signal to continuously calculate the Standard Deviation of the values within specified time intervals (e.g. every hour). This operation uses Welford's online algorithm. Intervals are based on the UTC time and allows specifying an offset. Note: Seasonal time changes are unknown to this operation.

Learn more about Welford's Online Algorithm: https://en.wikipedia.org/wiki/Algorithms_for_calculating_variance#Welford's_online_algorithm
Intervals are based on the UTC time and allows specifying an offset.
Note: Timezones and seasonal time changes are unknown to these operation functions.

Configuration

I/O	Details
Input	Any Numeric signal may be used with this transform function.
Output 1 - Standard Deviation	The standard deviation for the current interval, default type is same as Input signal
Output 2 - Average	The average for the current interval, default type is same as Input signal
Output 3 - Count	The count of values within the current interval, type `Number`
Output 4 - M2	The squared distance from the mean, type `Number`

Parameters:

Parameter	Format	Required	Description
Interval Size	Enum (a time interval)	✔︎	Selectable Time Interval for operation. (10 minutes, 30 minutes, 1 hour, 24 hours )
Interval Alignment	Enum (an offset time period)	✔︎	Selectable Offset from the hourly UTC time for the Interval. (0, 15m, 30m, 45m, 1h, 2h, ..., 23h)

Signal History is Not Required to reduce storage usage

To save on signal storage history, the output signal history property may be disabled for any outputs that do not need history to be viewed or reported on. History of any of the output signals is not required for this function.

Sample Standard Deviation in Time Interval¶

Operates on a signal to continuously calculate the Standard Deviation of the values within specified time intervals (e.g. every hour). This operation uses Welford's online algorithm. Intervals are based on the UTC time and allows specifying an offset. Note: Seasonal time changes are unknown to this operation.

Learn more about Welford's Online Algorithm: https://en.wikipedia.org/wiki/Algorithms_for_calculating_variance#Welford's_online_algorithm
Intervals are based on the UTC time and allows specifying an offset.
Note: Timezones and seasonal time changes are unknown to these operation functions.

Configuration

I/O	Details
Input	Any Numeric signal may be used with this transform function.
Output 1 - Sample Standard Deviation	The sample standard deviation for the current interval, default type is same as Input signal
Output 2 - Average	The average for the current interval, default type is same as Input signal
Output 3 - Count	The count of values within the current interval, type `Number`
Output 4 - M2	The squared distance from the mean, type `Number`

Parameters:

Parameter	Format	Required	Description
Interval Size	Enum (a time interval)	✔︎	Selectable Time Interval for operation. (10 minutes, 30 minutes, 1 hour, 24 hours )
Interval Alignment	Enum (an offset time period)	✔︎	Selectable Offset from the hourly UTC time for the Interval. (0, 15m, 30m, 45m, 1h, 2h, ..., 23h)

Signal History is Not Required to reduce storage usage

To save on signal storage history, the output signal history property may be disabled for any outputs that do not need history to be viewed or reported on. History of any of the output signals is not required for this function.

Rolling Aggregation by Count¶

Rolling aggregation of past signal values by count.

Configuration

I/O	Details
Inputs	1 Numeric signal
Outputs	1 Numeric signal (Type / Unit can be configured)

Parameters:

Parameter	Format	Required	Description
Number of past values to combine	Number	✔︎	Number of values to continuously combine with each new value as a rolling aggregation
How to combine the values	Enum operations	✔︎	Operation for aggregating the data (min, max, mean, sum )

Deprecated Windowed Transforms¶

Deprecated Off the Shelf Transforms

The following transforms have at a time been available off the shelf to use. These have been deprecated and are no longer available, but some may still be in use. It's recommended to migrate to using the supported functions above.

Aggregation by Time Interval¶

Deprecation Notice

This function has been deprecated for further use. Contact Exosite for alternative options.

Aggregates signal values over the time window specified.

Configuration

I/O	Details
Inputs	1 Numeric signal
Outputs	1 Numeric signal (Type / Unit can be configured)

Parameters:

Parameter	Format	Required	Description
Interval Size	Number	✔︎	Interval in seconds for the time window for aggregation
Interval Alignment	Number	✔︎	Alignment in Seconds since Unix Epoch to align intervals against
How to combine the values	Enum operations	✔︎	Operation for aggregating the data (min, max, mean, sum, count )

Rolling Aggregation by Duration¶

Deprecation Notice

This function has been deprecated for further use. Contact Exosite for alternative options.

Rolling aggregation of past signal values in the past time duration (in seconds).

Configuration

I/O	Details
Inputs	1 Numeric signal
Outputs	1 Numeric signal (Type / Unit can be configured)

Parameters:

Parameter	Format	Required	Description
Duration in seconds	Number	✔︎	Time duration to continuously combine as a rolling aggregation
How to combine the values	Enum operations	✔︎	Operation for aggregating the data (min, max, mean, sum )