With Data Warehouse integrations like BigQuery and Redshift, Toolio is syncing its internal database with the integrated datawarehouse.

As is the case with each syncing job, the 2 key questions are sync frequency and pagination approach.

Sync frequency is typically configured to be once a day, and you can configure at what time this sync is triggered. You can configure this frequency under Settings > Integrations

We recommend that you set the integration to run at the beginning of a day, when all the data from the previous day's transactions have been populated in your data warehouse. Otherwise, you risk seeing incomplete days' data within Toolio.

Pagination refers to identifying chunks of data to go through as Toolio is syncing the data. The best way to approach pagination, is through adding an updatedAt field in your data warehouse. This way Toolio can go through your data in the order the data was last updatedAt.

If there is no updatedAt field, Toolio would either have to do a full data sync each time it runs (which is not sustainable for larger tables like sales and inventory), or fetch the data for the last N # of days based on timestamp. Below are the default and the recommended approaches within Toolio