Skip to main content
All CollectionsIntegrationsMagento Open Source and Adobe Commerce
Developer settings for Magento Open Source and Adobe Commerce
Developer settings for Magento Open Source and Adobe Commerce

A reference of developer settings found in Magento Open Source and Adobe Commerce.

Gareth Burroughes avatar
Written by Gareth Burroughes
Updated over 3 weeks ago

Import settings

Use this section to tune the rate and resource utilisation of bulk syncs according to the volume of data you need to sync and the capacity of your system. We have set some default configurations that balance speed and efficiency.


Understand continuous sync

Bulk contact (Customer, Guest, Subscriber) and Insight data syncs (Order, Catalog) run continuously. When the relevant cron job runs, the sync processes all available rows in batches and stops only when everything has been synced.

Example: Syncing 100,000 customers with the default configuration:

  • ddg_automation_customer_sync starts

  • Customer IDs fetched from the email_contact table in batches of 500

  • 4 batches of 500 = 2000 - time to create a mega batch

  • Sync sends data to Dotdigital in 50 mega batches of 2000 each

Tips:

  • Observe the module's connector.log to see batch counts and sync progress.

  • It's normal for a single cron job to be in a running state for longer than the configured cron cadence. Equally, it is normal for subsequently scheduled cron jobs to be missed because a previous one is already busy syncing.

  • Review and Wishlist syncs do not run continuously. They still export a single batch of data, send it to the email_importer table, and wait for the next cron execution. The size of these syncs' data batch is still dictated by the Transactional data batch size.

Batch size vs mega batch size

Continuous sync means a continual cycle of:

  • query database > export data (get a single ‘batch’)

  • query database > export data

  • query database > export data

  • reach the mega batch size limit

  • send a mega batch to Dotdigital

  • store the mega batch as JSON in the email_importer table

  • repeat

A mega batch is, by default, the result of multiple database queries.


Transactional data break

Setting a break value is a way to limit the work done in any single cron sync execution. By default, it is not set to expedite fast data transfer. If you are concerned about resource utilisation, use the break value to sync data in manageable chunks.

If the break value is set, continuous sync stops when the total number of processed rows exceeds this. The sync then finishes and waits for the next cron run to process more rows. Setting break value applies to all continuous syncs; it effectively makes them non-continuous.

Example: Syncing 100,000 customers with break value set to 20,000:

  • Cron process begins

  • 5 mega batches are sent to Dotdigital

  • Sync stops and waits for the next cron execution


Common issues with cron processes and database queries

  • Overloaded server due to long-running cron processes

    Set the Transactional data break to address this issue. Although the sync rate slows down, it allows for monitoring without the concern of runaway processes.

  • Impact of excessive database queries on the system
    To mitigate this, reduce the Contact sync batch size to limit the number of customers data is being exported for in any one loop. Un-map the most query-hungry data fields in Dotdigital > Data Mapping, starting with:

    • MOST_PUR_CATEGORY

    • MOST_PUR_BRAND

    • FIRST_BRAND_PUR

    • LAST_BRAND_PUR

  • Cron Tasks Stuck in running for an extended time
    This is expected behaviour, as syncs run continuously by default until all eligible data is processed.

  • Dotdigital sync crons showing as Missed

    This is also expected behaviour. A cron process may over-run its scheduled window if it is processing large amounts of unprocessed or unimported data. Magento's cron scheduler only spawns one running job of each type at a time, and marks subsequent jobs as Missed until the running job is finished.

  • Imports failing due to unserialized errors or bad JSON data
    This issue likely occurs when the sync attempts to save too much data into the import_data column fails. To resolve this, adjust the relevant mega batch size. This problem should occur less frequently after v4.27.0.

Tips

  • Reducing single batch size configs to low numbers can be counter-productive, since it results in a proportionally larger number of database queries.

  • Dotdigital accepts up to 50MB of JSON in a single import request. This is a lot of JSON, and trying to maximise mega batch size to meet this limit is not advised.

    Depending on the number of data fields you have chosen to sync, or the number of order and product attributes you have selected, 50MB might equate to 100,000-150,000 rows. Moving this much data around, serialising and un-serialising it, reading and writing it, is slow and may lead to timeouts.


Sync settings

Automap data fields

If you don't want to manually map the key information that you store on customers from your website to Dotdigital contact data fields, then you can use this option to do it automatically.

To automatically map your data fields:

  1. Go to Stores > Configuration > DOTDIGITAL > Developer and find the heading Admin Task.

  2. For Automap Datafields, select Run Now. All relevant contact data fields are immediately created in your Dotdigital account to match your customer, address, and sales information fields.

Delete all contact IDs

You might want to synchronise your store with a different Dotdigital account, or even synchronise a different store front to a new Dotdigital account. To ensure this works correctly, you need to delete all the customer contact IDs that have been stored against the customers, as these are unique to the Dotdigital account.

To do this:

  1. Go to Stores > Configuration > DOTDIGITAL > Developer and find the heading Admin Task.

  2. For Delete All Contact Ids, select Run Now. All contact IDs are immediately deleted from your account.

Manually run sync

Instead of waiting for the next automatic contact data synchronisation in your account, you can manually run one instead.

To run a manual sync:

  1. Go to Stores > Configuration > DOTDIGITAL > Developer and find the heading Manually Run Sync.

  2. For each contact data you want run a sync for, select Run Now. Synchronisation immediately starts.

    Customer, Subscriber, Order, Catalog, Wishlist syncs create a batch, but do not import it. You need to select Importer sync to import the batch after creating it.

Setting

Description

Customer Sync

Syncs your customers to your mapped Dotdigital list.

Subscriber Sync

Syncs your subscribers to your mapped Dotdigital list.

SMS Subscriber Sync

Syncs your SMS subscribers to your mapped Dotdigital list.

Guest Sync

Syncs your guests to your mapped Dotdigital list.

Order Sync

Syncs your order data to your Dotdigital account.

Wishlist Sync

Syncs your wishlist data to your Dotdigital account.

Review Sync

Syncs your review data to your Dotdigital account.

Catalog Sync

Syncs your catalog data to your Dotdigital account.

Importer Sync

Syncs your importer data to your Dotdigital account.

Template Sync

Syncs your template data to your Dotdigital account.

Reset Sync

You can refresh all of the customer data stored in Dotdigital that you've already imported from your store, marking it all for import again and ensuring it's up to date.

To reset your customer data:

  1. Go to Stores > Configuration > DOTDIGITAL > Developer and find the heading Reset Sync Options.

  2. For Date Range, select the From and To dates you want to reset your customer data from.

    If no date range is set then a full data refresh runs.

  3. For each contact data type you want reset, select Run Now. The reset immediately starts.

Setting

Description

Reset Contacts

Resets the contact data in your account from the data range.

Reset Subscribers

Resets the subscriber data in your account from the data range.

Reset SMS Subscribers

Resets the SMS subscriber data in your account from the data range.

Reset Order Data

Resets the order data in your account from the data range.

Reset Review Data

Resets the review data in your account from the data range.

Reset Wishlist Data

Resets the wishlist data in your account from the data range.

Reset Catalog Data

Resets the catalog data in your account from the data range.

Reset B2B Quotes

Resets the B2B quotes in your account from the data range.

Enable Subscribers Sales Data Sync

This setting can be enabled to sync subscribers’ sales data to Dotdigital.

Enabling this setting increases server load and overall sync times.

  1. Expand the Enable Subscribers Sales Data Sync drop-down menu and select Yes.

  2. Select Save Config.

Enable Remove /pub directory from URLs

This setting can be enabled to remove the /pub directory from image URLs in catalog sync and insight data:

  1. Expand the Remove /pub directory from URLs drop-down menu and select Yes.

  2. Select Save Config.


Enable debug mode on your website

  1. Expand the Debug Mode drop-down menu and select Enable.

  2. For Debug Api Calls Time, enter a number of seconds.

    Any API calls that take longer than the number entered to run are logged.

  3. Select Save Config.


OAUTH Credentials

  1. Find the credentials you want to update and enter valid credentials.

  2. Select Save Config.

Setting

Description

Client ID

Enter your Client ID.

Client Secret Key

Enter your Client Secret Key.

Connection

Select Connect to open your connected Dotdigital account.

Custom OAUTH Domain

Enter the custom domain for authorisation.

Make sure that the base URL ends with / (slash), for example. https://yourdomain/magento/

Custom Authorization URL

Enter the custom authorisation URL.

Make sure that the base URL ends with / (slash), for example. https://yourdomain/magento/


Dynamic Pages IP Restriction

When you install the Dotdigital for Adobe Commerce connector, we generate a unique passcode that is used to access external dynamic content. By using this passcode, you can be sure that basket contents and coupon codes can be viewed only if that passcode is sent with the GET request to the external dynamic content URL, adding security.

You can also enter a list of requesting IP addresses that these URLs display for when requested from them:

  1. For Dynamic Pages IP Restriction, enter an IP address.


    If you want to enter multiple IP addresses, make sure to separate them with a comma and a space. For example, 52.174.92.164, 20.54.90.19.


    Make sure you keep the latest dotdigital IP ranges included.
    Learn more in IP addresses used by Dotdigital.

  2. Select Save Config.


Manage Cron Timings

To set how frequently you want cron jobs to run:

  1. Expand the drop-down menu for the cron timing you want to edit and choose a time interval.

  2. Select Save Config.

Setting

Options

Importer

  • Every 5 Minutes

  • Every 10 Minutes

  • Every 15 Minutes

Customer

  • Every 1 minute

  • Every 5 minutes

  • Every 10 minutes

  • Every 15 Minutes

  • Every 30 Minutes

  • Every 60 Minutes

Guest

  • Every 1 minute

  • Every 5 minutes

  • Every 10 minutes

  • Every 15 Minutes

  • Every 30 Minutes

  • Every 60 Minutes

Subscriber

  • Every 1 minute

  • Every 5 minutes

  • Every 10 minutes

  • Every 15 Minutes

  • Every 30 Minutes

  • Every 60 Minutes

Order

  • Every 1 minute

  • Every 5 minutes

  • Every 10 minutes

  • Every 15 Minutes

  • Every 30 Minutes

  • Every 60 Minutes

Catalog

  • Every 1 minute

  • Every 5 minutes

  • Every 10 minutes

  • Every 15 Minutes

  • Every 30 Minutes

  • Every 60 Minutes

Review And Wishlist

  • Every 1 minute

  • Every 5 minutes

  • Every 10 minutes

  • Every 15 Minutes

  • Every 30 Minutes

  • Every 60 Minutes

Consent

  • Disabled

  • Every 15 Minutes

  • Every 30 Minutes

  • Every 60 Minutes

Cleaner

  • Every 30 Days

  • Every 14 Days

  • Every 7 Days

  • Every Day


System alerts

System alerts can notify you when certain functions go wrong in your account.

Enable system messages

When enabled, this setting allows you to send in-app system messages to admin users when errors that may affect the Dotdigital extension are detected:

  1. Expand the Enable System Messages drop-down menu and select Yes.

  2. Select Save Config.

Enable email notifications

When enabled, this setting allows you to send system alert email notifications when errors that may affect the Dotdigital extension are detected:

  1. Expand the Enable Email Notifications drop-down menu and select Yes.

  2. For Select Roles, choose which users you want email notifications to be sent to.
    You can select multiple roles by holding the Ctrl key (Windows) or Cmd key (Mac) while selecting.

  3. Expand the Alert Frequency drop-down menu and select a time period.


    This setting controls both email notification frequency and the time period in which the error is reported. For example, select 24 Hours to receive an email every 24 hours, reporting any errors that may have occurred in the last 24 hours.

  4. Select Save Config.


PWA settings

Use this setting to specify if you use a PWA storefront URL that is different from your account’s default frontend base URL:

  1. For PWA Storefront Base URL, enter the URL of your PWA storefront, for example, https://pwa.yourdomain.com/.

  2. Select Save Config.

Did this answer your question?