Skip to main content

Set up a data firehose (public preview)

Learn to set up a data firehose to your external data storage.

Gareth Burroughes avatar
Written by Gareth Burroughes
Updated this week

Public preview

This feature is currently in public preview to CXDP customers. Apply to join the preview.

Exporting your event data lets you analyse and use your marketing data more effectively, providing deeper insights and helping you to make data-driven decisions.

Data sync frequency

Data firehose offers three options for sync frequency: every 15 minutes, hourly, or daily.


1. Connect a data firehose

  1. In Dotdigital, go to Connect.

  2. Under Data management, select Data firehose.

  3. Select NEW DATA FIREHOSE, then choose your data transfer method.
    Options:

    • SFTP/FTPS

    • S3: Amazon S3 storage

  4. Enter your SFTP or FTP details into the connect form.
    See the requested form fields:

  5. Select CONNECT.

Amazon S3 form fields

Label

Description

Bucket name

The name of the S3 bucket where the data will be stored.

Region endpoint

The AWS region where the S3 bucket is located.

Folder path

The specific folder path within the S3 bucket where the data will be stored.

Add accountID suffix

A suffix to be added to the account ID for identification purposes.

Access key

The access key for authenticating with AWS.

Secret access key

The secret access key for authenticating with AWS.

SFTPS form fields

Label

Description

FTP type

The type of FTP connection.

Host

The hostname or IP address of the SFTP server.

Port

The port number for the SFTP connection.

Folder path

The specific folder path on the SFTP server. Example: /

Add accountID suffix

A suffix to be added to the account ID for identification purposes.

Connect with

The method of authentication for the SFTP connection. Options: Login or Private key

Username

The username for authenticating with the SFTP server.

Password

The password for authenticating with the SFTP server.

Private key

The private key for authenticating with the SFTP server.

FTPS form fields

Label

Description

FTP type

The type of FTP connection. Example: FTPS

Host

The hostname or IP address of the FTPS server.

Port

The port number for the FTPS connection.

Folder path

The specific folder path on the FTPS server.

Add accountID suffix

A suffix to be added to the account ID for identification purposes.

Username

The username for authenticating with the FTPS server.

Password

The password for authenticating with the FTPS server.


2. Choose events to firehose

Next, you must choose the data for the events you want to sync. There are two options here: Engagement or Campaign.

  1. To sync event data, select the checkbox next to the data type you want to sync.

  2. Select NEXT.

Event types

Learn about all available event types in Dotdigital Data firehose event schema.


3. Choose a frequency

At this step, you choose how often you want to sync your data.

  1. Expand the Frequency drop-down menu, then choose a frequency.
    Options:

    • Every 15 minutes

    • Hourly

    • Daily

  2. Expand the Sync failure notification drop-down menu and choose how you would like to be notified if a sync fails.
    Options:

    • None

    • Email

    • In-app notification

    • Email and in-app notification

  3. Select NEXT.


4. Confirm your firehose setup

Finally, you must review and confirm you firehose the setup.

  1. Check the confirmation details displayed on this page.

  2. Select SYNC.

Did this answer your question?