The BulkExport API provides an efficient mechanism for retrieving all of your activity logs from the Twilio platform on an ongoing basis, or for one-off downloads. Using BulkExport, you can provide daily dumps of the previous day's Messages, eliminating the need to iterate through the Message list resource one page at a time to download your Message records. The BulkExport files are automatically removed after 7 days. You can also use BulkExport to regenerate daily dumps from previous days, which will generate a new set of daily dumps for your historical date range.
When a BulkExport completes, it can fire a Webhook and/or send an email. This applies to the daily export, if configured, and the custom jobs. In both cases, successful jobs will generate a Day resource for each day requested.
BulkExport is useful if you have:
- A data warehouse where you want to load messaging traffic to perform analysis
- A compliance store where messaging should be stored so that you have a permanent record
- Requirements to produce data
Control over BulkExports, and the data exported is available under the following URL:
There are 4 parts to BulkExport:
- a configuration that allows customers of Twilio Edition products to set automatic exports for each day
- a way to export a day's traffic
- managing exports in progress
- a list of exported days with their output files
The export configuration let you set, if you have the Twilio Editions product, whether exports will be done automatically for each day. This makes getting the data much simpler, and you can just configure the webhook. This will be called when the day's file is available, and you can then download and process your data.
Jobs let you choose a date range to export. You supply a date range, and one file will be generated for each day. The days are visible in the list of exported days. You will receive a callback and/or email for each job that completes. When a custom job completes, it will generate one Day Resource for each day in the time period requested as part of the custom job.
Jobs run and can be deleted before they are complete. Completed jobs will appear in this list for 7 days.
This resource lets you look at the list of export days available - either because they have been generated using an export job, or generated using the automatic daily export. Each day file is named with the ISO 8601 date, and days are listed in order of generation.
There is no way to delete completed Day files. They will be deleted automatically 7 days after their creation.
|Export Configuration||Configuration of export webhooks, and for Twilio Enterprise Edition customers, setting daily export|
|Export Custom Job||Creating new jobs that export files of data, and listing the current jobs and status. Running a job will create the corresponding Days.|
|Jobs||Viewing the status and deleting incomplete custom jobs to create exports.|
|Days||Listing and Fetching the exported days of message data|
To use BulkExport, you have two choices:
- If you have the Twilio Enterprise Edition, you can enable the daily export. This will generate one file per day automatically after the day is completed. This will occur in the 12 hours following the end of the UTC day.
- Create custom jobs using the Export Custom Job Resource
Then, you can list the completed days of export using the Day Resource then download the complete days.
The messages file format is a sequence of JSON records, similar to the Message Resource. The differences are:
- Timestamps are in UTC in ISO 8601 format
- The following fields are not present:
- price unit
- error message (but error_code is)
- subresource_uris (containing the media urls)