Configuring TSDB Storage¶
The EnOS Time Series Database (TSDB) enables you to store and retrieve important and frequently-accessed business data and provides specific storage types for different types of business data. Each storage type has specific data storing and reading capability. You can define customized data storage policies based on your business needs. Currently, EnOS TSDB provides the following storage types:
- Formatted data storage: Formatted data is the core, defined by models and measurement points, and needs to be stored in various data buckets of TSDB for subsequent viewing, acquisition and analytical processing. The supported types of formatted data storage are as follows.
- AI Raw Data
- AI Normalized Data
- DI Data
- PI Data
- Generic Data
- Unformatted data storage: For the raw data sent directly from the device side, some of them are non-core, without model and measurement point definition, and cannot be stored in the various data buckets of TSDB, but these data need to be stored, viewed and accessed, and may be used for analysis or standardization later, so special buckets are needed to store this part of Unformatted data.
Configure Formatted Data Storage Policies¶
Log in to the EnOS Management Console, click Time Series Data Management > Storage Policies, and select Formatted record to go to the Formatted Data Storage Policies page.
Note
To avoid the lack of system resources and system instability, the function of modifying storage time will not be available after 2.3CU5. Please contact your system administrator if you are required to make changes.
Configuring the AI Raw Data Storage Type¶
AI (analogy input) data consists of physical parameters like temperature, pressure, flow, etc. The values of these parameters are collected by various sensors, converted to electric signals by a transmitter, and transferred to the analogy input of the controller. To reflect real-time data changes, AI data generally has high ingestion frequency, usually by seconds, which results in having a huge data size. In most business cases, the second-level raw data is usually normalized by minute-level processing, so this kind of second-level raw data does not need to be stored for a long time.
The features of the AI Raw Data storage type are as follows.
| Feature | Description |
|---|---|
| Data type | AI type data only (the data type is selected when defining a measurement point). |
| Data source | The measurement point raw data uploaded to the cloud (or normalized data processed by the streaming engine). |
| Storage time | 3 Months by default. |
| API for reading data | Get Asset AI Raw Data (gets the stored AI raw data of certain measurement points of specified assets within a time range). |
Configure the AI Raw Data storage type with the following steps.
Log in to the EnOS Management Console, click Time Series Data Management > Storage Policies, and select the created storage policy group.
Move the cursor on the AI Raw Data storage type and click the Edit icon to open the Edit Storage Policies page.
Select models and the corresponding measurement points for which the data storage policy serves.
Click OK to save the storage policy.
Configuring AI Normalized Data Storage Type¶
As described in the above section, the second-level raw data is usually normalized by minute-level processing in most business cases to reduce the data size. EnOS TSDB stores the normalized data separately and supports extra processing for the stored data. When you retrieve the minute-level normalized data with API, EnOS TSDB provides aggregation functions for further data processing.
The features of the AI Normalized Data storage type are as follows.
| Feature | Description |
|---|---|
| Data type | AI type data only (the data type is selected when defining a measurement point). |
| Data source | Normalized data processed by the streaming engine (or measurement point raw data uploaded to the cloud). |
| Normalization processing | If the data source is AI raw data, the second-level suffix of the data timestamp will be removed when the data is stored in TSDB. Therefore, only the last-coming data record of one minute will be stored. |
| Storage time | 3 Months by default. |
| API for reading data | Get Asset AI Data with Aggregation Logic (getting the stored minute-level normalized AI data of certain measurement points of specified assets in a time range). |
| Aggregation functions | TSDB provides aggregation functions for further processing the stored data when retrieving data with the Get Asset AI Data with Aggregation Logic API. The supported aggregation functions are cnt, avg, sum, max, min, first, and last. Note that if the data aggregation logic is NOT used, the value of the interval parameter must be 0. |
The configuration of the AI Normalized Data storage type is similar with that of the AI Raw Data storage type.
Configuring DI Data Storage Type¶
DI (digital Input) data is usually used for recording device status. For example, a sensor outputs the device running status, such as water flow switch and wind speed switch where the status is transferred to the controller and then converted to the digital value 1 or 0, which can be used for logic analysis and computation. DI data is usually enumerable status values, and the frequency of the data upload is not fixed. Data is generally uploaded or stored only when the device status changes. For DI data, EnOS TSDB provides separated storage and reading policy.
The features of the DI Data storage type are as follows.
| Feature | Description |
|---|---|
| Data type | DI type data only (the data type is selected when defining a measurement point). |
| Data source | The measurement point raw data uploaded to the cloud. |
| Storage time | 3 Months by default. |
| API for reading data | Get Asset DI Data (gets the status change data of specified assets within a time range). |
The configuration of the DI Data storage type is similar with that of the AI Raw Data storage type.
Configuring PI Data Storage Type¶
PI (pulse input) data is usually used for power meter readings. For PI data, EnOS TSDB provides separated storage and reading policy.
The features of the PI Data storage type are as follows.
| Feature | Description |
|---|---|
| Data type | PI type data only (the data type is selected when defining a measurement point). |
| Data source | Minute-level normalized PI data (if the measurement point data is second-level raw data, the suffix of the data timestamp will be removed when the data is stored in TSDB). |
| Storage time | 3 Months by default. |
| API for reading data |
|
The configuration of the PI Data storage type is similar with that of the AI Raw Data storage type.
Configuring Generic Data Storage Type¶
When configuring device models, you can choose AI, DI, PI, or Generic as the data type of a measurement point. EnOS TSDB provides a separate storage policy for generic data.
The features of the Generic Data storage type are as follows.
| Feature | Description |
|---|---|
| Data type | Generic type data only (the data type is selected when defining a measurement point). |
| Data source | The measurement point raw data uploaded to the cloud. |
| Storage time | 3 Months by default. |
| API for reading data | Get Asset Generic Data (gets the generic data of certain measurement points of specified assets within a time range.). |
The configuration of the Generic Data storage type is similar with that of the AI Raw Data storage type.
Configure Unformatted Data Storage Policies¶
Follow these steps to configure an unformatted data store.
- Log in to the EnOS Management Console, click Time Series Data Management > Storage Policies, and select Unformatted record to go to the Unformatted Data Storage Policies page.
- Move the cursor on the Unformatted record storage type and click the Edit icon to open the Edit Storage Policies page.
- Click OK to save the storage policy.