Skip to content

Historical Data

Data Retention

The Data Retention setting refers to the duration for which data is stored for each resource created and is applicable across all provisioned devices. Your published data is retained in the database until the specified retention period is met. If the server's published time surpasses the retained duration, the data is archived and deleted from database. To modify the retention duration, navigate through: Resource -> Resource Edit -> Data storage.

Note

By default, the data retention period is set to 12 months, with a minimum of 3 months and a maximum retention period up to 120 months.

Historical Data Query

By User Interface

You can access and query published historical data via the Connectivity Platform User Interface. To do this, follow the steps below:

  1. Go to 'Signals' and select a resource or signal of interest.
  2. Set the 'Start Time' and 'End Time' for the period.
  3. Click 'APPLY' to view the results.

For future reference or deeper analysis, these results can be exported as a CSV file by clicking on 'EXPORT'.

By Lua script operation

The Device2 service comprises multiple APIs aimed at querying historical data. For more in-depth information, please refer to the Device2 script operations under the category "DATA QUERY".

Archive Data

The historical data will be transferred to Content Storage for archiving. This archiving process happens daily for all Resource and Signal data.

Each archive file is saved in the following format: daily_archive/year=####/month=##/day=##/<solution id>_<year>_<month>_<day>_<number of file>.gzip. It's then stored securely within Content Storage for later access and use.

Note

Each archived file has a maximum size limit of 100 MB. If the volume of data handled within a single day is substantial, the archival process will automatically divide the data into several smaller files. Each file will be distinguished by a specific file number for efficient management and retrieval.

Download Archive Data File

You can download the published historical data file via the Connectivity Platform User Interface(UI). To do this, follow the steps below:

  1. Go to your connector 'Content' page.
  2. Select File Types to "Signal Archives".
  3. Set the 'Start Time' and 'End Time' for the period.
  4. Click 'Download' of specific file.

Archive Data Restore

Data Restore

The Connectivity Platform supports the restoration of archived data via the User Interface (UI). Users can restore device data using several options.

To do this, follow the steps below:

  1. Go to your connector 'Signals' page.
  2. Click the 'ARCHIVES' button to review the list of archived data files.
  3. Click 'RESTORE SIGNAL' button to access Restore Signal Archives page.
  4. Define the data restore time range or paste the archive file's URL (copied from the list of archived data files).
  5. (Optional) If you only need to restore specific signal data for a specific device, enter the device name and signal name separately.
  6. Write the reason behind this restoration.
  7. Click 'SUBMIT' to send the request.
Note

By default, restored data will be retained for 30 days, regardless of any new data retention settings.
If a new retention policy is required for restored data, you can enable 'New Data Retention' and set a new policy. Please note that this new retention period will be calculated from the signal's reported time.If the reported time plus the new retention policy is still in the past, the restored data will be retained for 30 days.

Restoration Requests Review

After submitting the Restoration Request, users can review the historical requests list through the User Interface (UI).

To do this, follow the steps below:

  1. Go to your connector 'Signals' page.
  2. Click the 'ARCHIVES' button.
  3. Click the 'RESTORATION REQUESTS' button to review all requests list.
  4. Click on a task row to review detailed information.