Skip to main content
All CollectionsIntegrationsMagento Open Source and Adobe Commerce
Developer settings for Magento Open Source and Adobe Commerce
Developer settings for Magento Open Source and Adobe Commerce

A reference of developer settings found in Magento Open Source and Adobe Commerce.

Gareth Burroughes avatar
Written by Gareth Burroughes
Updated yesterday

Import settings

Use this section to tune the rate and resource utilisation of bulk syncs according to the volume of data you need to sync and the capacity of your system. We have set some default configurations that balance speed and efficiency.

Understanding continuous sync

Bulk contact (Customer, Guest, Subscriber) and insight data syncs (Order, Catalog) run continuously. When the relevant cron job runs, the sync will process all available rows in batches and stop only when everything has been synced.

Example: Syncing 100,000 customers with the default configuration:

  • ddg_automation_customer_sync starts

  • Customer IDs fetched from the email_contact table in batches of 500

  • 4 batches of 500 = 2000 - time to create a mega batch

  • Sync sends data to Dotdigital in 50 mega batches of 2000 each

Tips:

  • Observe the module's connector.log to see batch counts and sync progress.

  • It's normal for a single cron job to be in a 'running' state for longer than the configured cron cadence. Equally, it is normal for subsequently scheduled cron jobs to be 'missed' because a previous one is already busy syncing.

  • Review and Wishlist syncs do not run continuously. They still export a single batch of data, send it to the email_importer table, and wait for the next cron execution. The size of these syncs' data batch is still dictated by the Transactional data batch size.

Batch size vs mega batch size

Continuous sync means a continual cycle of:

  • query database > export data (get a single ‘batch’)

  • query database > export data

  • query database > export data

  • reach the mega batch size limit

  • send a mega batch to Dotdigital

  • store the mega batch as JSON in the email_importer table

  • repeat

A mega batch is, by default, the result of multiple database queries.

Transactional data break

Setting a break value is a way to limit the work done in any single cron 'sync' execution. By default, it is not set to expedite fast data transfer. If you are concerned about resource utilisation, use the break value to sync data in manageable chunks.

If the break value is set, continuous sync will stop when the total number of processed rows exceeds this. The sync then finishes and waits for the next cron run to process more rows. Setting break value applies to all continuous syncs; it effectively makes them non-continuous.

Example: Syncing 100,000 customers with break value set to 20,000:

  • Cron process begins

  • 5 mega batches are sent to Dotdigital

  • Sync stops and waits for the next cron execution

Common issues with cron processes and database queries

Overloaded server due to long-running cron processes

Set the Transactional data break to address this issue. Although the sync rate will slow down, it allows for monitoring without the concern of runaway processes.

Impact of Excessive Database Queries on the System

To mitigate this, reduce the Contact sync batch size to limit the number of customers being exported data for in any one loop. Unmap the most query-hungry data fields in Dotdigital > Data Mapping, starting with:

  • MOST_PUR_CATEGORY

  • MOST_PUR_BRAND

  • FIRST_BRAND_PUR

  • LAST_BRAND_PUR

Cron Tasks Stuck in running for an extended time:

  • This is expected behavior, as syncs run continuously by default until all eligible data is processed.

Dotdigital sync crons showing as 'missed'

This is also expected behavior. A cron process may over-run its scheduled window if it is processing large amounts of un-processed or not-imported data. Magento's cron scheduler will only spawn one running job of each type at a time and will mark subsequent jobs as 'Missed' until the running job is finished.

Imports failing due to unserialized errors or bad JSON data

This issue likely occurs when the sync attempts to save too much data into the import_data column fails. To resolve this, adjust the relevant mega batch size. This problem should occur less frequently after v4.27.0.

  • Reducing single batch size configs to low numbers can be counter-productive since it will result in a proportionally larger number of database queries.

  • Dotdigital will accept up to 50MB of JSON in a single import request. This is a lot of JSON, and trying to maximise mega batch size to meet this limit is not advised. Depending on the number of data fields you have chosen to sync, or the number of order and product attributes you have selected, 50MB might equate to 100,000-150,000 rows. Moving this much data around, serializing and unserializing it, reading and writing it, will be slow and may lead to timeouts.

Tips

  • Reducing single batch size configs to low numbers can be counter-productive, since it will result in a proportionally larger number of database queries.

  • Dotdigital will accept up to 50MB of JSON in a single import request. This is a lot of JSON, and trying to maximise mega batch size to meet this limit is not advised. Depending on the number of data fields you have chosen to sync, or the number of order and product attributes you have selected, 50MB might equate to 100,000-150,000 rows. Moving this much data around, serializing and unserializing it, reading and writing it, will be slow and may lead to timeouts.


Sync settings

Automap data fields

If you do not want to manually map the key information that you store on customers from your website to Dotdigital contact data fields, then you can use this option to do it automatically.

To automatically map your data fields:

  1. Go to Stores > Configuration > DOTDIGITAL > Developer and find the heading Admin Task.

  2. For Automap Datafields, select Run Now. All relevant contact data fields are immediately created in your Dotdigital account to match your customer, address, and sales information fields.

Delete all contact IDs

If you want to synchronise your store with a different Dotdigital account, or even synchronise a different store front to a new Dotdigital account. To ensure this works correctly you need to delete all the customer contact IDs that have been stored against the customers, as these are unique to the Dotdigital account.

To delete all contact IDs:

  1. Go to Stores > Configuration > DOTDIGITAL > Developer and find the heading Admin Task.

  2. For Delete All Contact Ids, select Run Now. All contact IDs are immediately deleted from your account.

Manually run sync

Instead of waiting for the next automatic contact data synchronisation in your account, you can manually run one instead.

To run a manual sync:

  1. Go to Stores > Configuration > DOTDIGITAL > Developer and find the heading Manually Run Sync.

  2. For each contact data you want run a sync for, select Run Now. Synchronisation immediately starts.

  3. Customer, Subscriber, Order, Catalog, Wishlist syncs create a batch, but do not import it. You need to select Importer sync to import the batch after creating it.

Setting

Description

Customer Sync

Syncs your customers to your mapped Dotdigital list.

Subscriber Sync

Syncs your subscribers to your mapped Dotdigital list.

SMS Subscriber Sync

Syncs your SMS subscribers to your mapped Dotdigital list.

Guest Sync

Syncs your guests to your mapped Dotdigital list.

Order Sync

Syncs your order data to your Dotdigital account.

Wishlist Sync

Syncs your wishlist data to your Dotdigital account.

Review Sync

Syncs your review data to your Dotdigital account.

Catalog Sync

Syncs your catalog data to your Dotdigital account.

Importer Sync

Syncs your importer data to your Dotdigital account.

Template Sync

Syncs your template data to your Dotdigital account.

Reset Sync

You can refresh all of the customer data stored in Dotdigital that you've already imported from your store, marking it all for import again and ensuring it's up to date.

To reset your customer data:

  1. Go to Stores > Configuration > DOTDIGITAL > Developer and find the heading Reset Sync Options.

  2. For Date Range, select the From and To dates you want to reset your customer data from.

  3. If no date range is set then a full data refresh runs.

  4. For each contact data type you want reset, select Run Now. The reset immediately starts.

Setting

Description

Reset Contacts

Resets the contact data in your account from the data range.

Reset Subscribers

Resets the subscriber data in your account from the data range.

Reset SMS Subscribers

Resets the SMS subscriber data in your account from the data range.

Reset Order Data

Resets the order data in your account from the data range.

Reset Review Data

Resets the review data in your account from the data range.

Reset Wishlist Data

Resets the wishlist data in your account from the data range.

Reset Catalog Data

Resets the catalog data in your account from the data range.

Reset B2B Quotes

Resets the B2B quotes in your account from the data range.

Enable Subscribers Sales Data Sync

This setting can be enabled to sync subscribers’ sales data to Dotdigital.

Enabling this setting increases server load and overall sync times.

  1. Expand the Enable Subscribers Sales Data Sync drop-down menu and select Yes.

  2. Select Save Config.

Enable Remove /pub directory from URLs

This setting can be enabled to remove the /pub directory from image URLs in catalog sync and insight data:

  1. Expand the Remove /pub directory from URLs drop-down menu and select Yes.

  2. Select Save Config.


Debug

Enable debug mode on your website

  1. Expand the Debug Mode drop-down menu and select Enable.

  2. For Debug Api Calls Time, enter a number of seconds.

    • Any api calls that take longer than the number entered to run are logged.

  3. Select Save Config.


OAUTH Credentials

  1. Find the credentials you want to update and enter valid credentials.

  2. Select Save Config.

Setting

Description

Client ID

Enter your Client ID.

Client Secret Key

Enter your Client secret key.

Connection

Select Connect to open your connected Dotdigital account.

Custom OAUTH Domain

Enter the custom domain for authorization.

Custom Authorization URL

Enter the custom authorization url.


Dynamic Pages IP Restriction

When you install the Dotdigital for Adobe Commerce connector, we generate a unique passcode that is used to access external dynamic content. By using this passcode you can be sure that basket contents and coupon codes can be viewed only if that passcode is sent with the GET request to the external dynamic content URL, adding security.

You can also enter a list of requesting IP addresses that these URLs will display for when requested from them:

  1. For Dynamic Pages IP Restriction, enter an IP address.

    1. If you want to enter multiple IP addresses, make sure to separate them with a comma and a space. For example, 52.174.92.164, 20.54.90.19.

  2. Select Save Config.


Manage Cron Timings

Set how frequently you want cron jobs to run:

  1. Expand the drop-down menu for the cron timing you want to edit and choose a time interval.

  2. Select Save Config.

Setting

Description

Options

Importer

Set how frequently you want the importer cron to run.

  • Every 5 Minutes

  • Every 10 Minutes

  • Every 15 Minutes

Customer

Set how frequently you want the customer cron to run.

  • Every 1 minute

  • Every 5 minutes

  • Every 10 minutes

  • Every 15 Minutes

  • Every 30 Minutes

  • Every 60 Minutes

Guest

Set how frequently you want the guest cron to run.

  • Every 1 minute

  • Every 5 minutes

  • Every 10 minutes

  • Every 15 Minutes

  • Every 30 Minutes

  • Every 60 Minutes

Subscriber

Set how frequently you want the subscriber cron to run.

  • Every 1 minute

  • Every 5 minutes

  • Every 10 minutes

  • Every 15 Minutes

  • Every 30 Minutes

  • Every 60 Minutes

Order

Set how frequently you want the order cron to run.

  • Every 1 minute

  • Every 5 minutes

  • Every 10 minutes

  • Every 15 Minutes

  • Every 30 Minutes

  • Every 60 Minutes

Catalog

Set how frequently you want the catalog cron to run.

  • Every 1 minute

  • Every 5 minutes

  • Every 10 minutes

  • Every 15 Minutes

  • Every 30 Minutes

  • Every 60 Minutes

Review And Wishlist

Set how frequently you want the review and wishlist cron to run.

  • Every 1 minute

  • Every 5 minutes

  • Every 10 minutes

  • Every 15 Minutes

  • Every 30 Minutes

  • Every 60 Minutes

Consent

Set how frequently you want the consent cron to run.

  • Disabled

  • Every 15 Minutes

  • Every 30 Minutes

  • Every 60 Minutes

Cleaner

Set how frequently you want the cleaner cron to run.

  • Every 30 Days

  • Every 14 Days

  • Every 7 Days

  • Every Day


System alerts

System alerts can notify you when certain functions go wrong in your account.

Enable system messages

When enabled, this setting allows you to send in-app system messages to admin users when errors that may affect the Dotdigital extension are detected:

  1. Expand the Enable System Messages drop-down menu and select Yes.

  2. Select Save Config.

Enable email notifications

When enabled, this setting allows you to send system alert email notifications when errors that may affect the Dotdigital extension are detected:

  1. Expand the Enable Email Notifications drop-down menu and select Yes.

  2. For Select Roles, choose which users you want email notifications to be sent to.

    • You can select multiple roles by holding the Ctrl key (Windows) or Cmd key (Mac) while selecting.

  3. Expand the Alert Frequency drop-down menu and choose a time period.

    • This setting controls both email notification frequency and the time period in which the error is reported. For example, select 24 Hours to receive an email every 24 hours, reporting any errors that may have occurred in the last 24 hours.

  4. Select Save Config.


PWA settings

Use this setting to specify if you use a PWA storefront URL that is different from your account’s default frontend base URL:

  1. For PWA Storefront Base URL, enter the URL of your PWA storefront.

  2. Select Save Config.

Did this answer your question?