We provide an optional service to export raw historical data to our customer's S3 buckets. This data is transferred in the form of parquet files. In order to use the data export feature, an organization must provide the name of an S3 bucket in the us-east-1 AWS region.
Every five minutes the backend checks for parquet files belonging to that organization and uploads them to the specified S3 bucket.
The data export process will automatically apply the correct scale and offset adjustments to all the values in a parquet file before uploading it to the customer's S3 bucket. The scale and offset values are fetched from the active CAN Databases associated with the device's model.
CAN signal names that contain " (double quote characters) are not supported by the data export feature and will be exported without any scale or offset adjustments.
The data export feature incurs additional charges.
When you process the exported parquet files, avoid dependencies based on the fastparquet library. We have found that it sometimes incorrectly parses values.