Unit 4: Managing and Processing Real-time Data¶
With your devices connected, your data starts to accumulate on EnOS. It could be quite a challenge to manage the vast amount of IoT data. Data Asset Managementprovides services that help you reduce your cost on data storage, lowers development barrier of stream analytics, and increases data access efficiency.
Before you dive into the offering, you’ll want to get yourself familiar with the key concepts about stream data processing and data storage.
Real-Time Data Flow¶
To be able to efficiently manage your data, you’ll first need to understand the Real-time Data Flow.
Configure Storage Policy¶
Note that the raw data uploaded to EnOS and the stream processing results are NOT stored when you don’t configure data storage policy to define which data goes where.
The storage policy gives you flexibility to determine what data to store, where to store, and how long to store. The design of EnOS data store takes best practices of energy/utility domain into consideration, by allowing you to store time-series data into different buckets according to the data types and provides different APIs for you to access data in different bucket to increase data access efficiency. For more information about data storage policy, see Configuring TSDB Storage.
Develop Stream Data Processing Jobs¶
To lower the development barrier of stream data processing, EnOS encapsulates the complexity of the underlying stream processing engine and provides GUI-based data development IDE to help you easily perform stream analytics.
For a deep dive into stream data processing on EnOS, see Stream Data Processing.
Get Data through APIs¶
As mentioned above, EnOS provides a data service API for each TSDB storage type. For more information about these APIs, go to EnOS Management Console > EnOS API.
Subscribe to Data¶
You can also use the Data Subscription Service to get your data more efficiently. The Data Subscription Service improves API calling efficiency by actively push data to the subscriber applications when new data arrives. The service allows you to subscribe to the following types of data:
You can also use the Data Subscription service to get your data more efficiently. The Data Subscription Service improves API calling efficiency by actively push data to the subscriber applications when new data arrives. The service allows you to subscribe to the following types of data: