The core data synchronisation process works by the creation and population of database tables during the installation process, and these are then updated and maintained using a handful of observers and cron services.
Before you start
Things you need to know:
The most accurate and up-to-date list of our cron tasks and their timings is our main crontab.xml file.
Creation of databases
Upon installation of the connector, we create new database tables. These are:
email_contact | All contacts to be synced to Dotdigital. |
email_order | All orders to be synced. |
email_wishlist | All wishlists to be synced. |
email_review | All reviews to be synced. |
email_catalog | All products to be synced. |
email_automation | All contacts due to be enrolled or that have been enrolled into a program. |
email_campaign | All campaign sends sent or due to be sent (abandoned cart or review requests only). |
email_importer | All batches of data that have been imported or are due to be imported. |
email_abandoned_cart | Stores quotes for the abandoned cart email series. |
email_contact_consent | Stores consent data if Stores > Configuration > Dotdigital > Configuration > Consent settings are enabled. |
email_coupon_attribute | Stores coupon properties for coupons created through our coupon URL. |
email_failed_auth | Logs authorisation failures when our dynamic content URLs are accessed with an incorrect passcode. |
email_rules | Stores exclusion rules configured in Marketing > Exclusion Rules. |
During installation, these tables become populated with your historical data, after which time they are updated automatically by cron services.
Cron services
The following cron services batch data from these tables and add them to the email_importer table, ready for import:
ddg_automation_customer_sync ddg_automation_guest_sync | These three jobs batch data from the email_contact table. Runs every 15 minutes by default. |
ddg_automation_subscriber_sync | Subscription state changes are handled by the |
ddg_automation_order_sync | Batches data from the email_order table. Runs every 15 minutes by default. |
ddg_automation_reviews_and_wishlist | Batches data from email_review and email_wishlist tables. Runs every 15 minutes by default. |
ddg_automation_catalog_sync | Batches product data from the email_catalog table. Runs every 15 minutes by default. |
All of the above cron services can have their frequency configured.
All of this data gets batched and then added to the email_importer table, which involves the running of the following cron service:
ddg_automation_importer | Looks for the oldest non-imported import and kicks off an import. This runs every 5 minutes (the frequency can be configured), checks the status of the last import, and, if complete, marks it as done and then starts the next one. |
The importer also checks the import fault file for all contact imports and, if it contains suppressed contacts, it updates their status in both the email_contact table and the core Adobe Commerce newsletter table.
The following cron services also perform the following tasks:
ddg_automation_abandonedcarts | Detects abandoned carts based on the time settings you specify and adds contacts that are due to receive an email to email_campaign table.
Runs every 5 minutes. |
ddg_automation_campaign | Creates contacts and performs sends of all unsent campaigns. Abandoned cart and review requests only.
Runs every 5 minutes. |
ddg_automation_status | Replaced by Message Queues. Removed in v4.24.0. |
ddg_automation_cleaner | Archives files created from the batching process.
This runs once a month, so it keeps your table clean. Contact Support if this isn't working. |
ddg_automation_monitor | Scans your tables for errors, to be flagged in System Alerts.
Runs every 15 minutes. |
ddg_automation_integration_insights | Sends data about your Adobe Commerce installation and your connector configuration to Dotdigital. Runs once a week. |
ddg_automation_unsubscribe | Fetches recent suppressions from Dotdigital and unsubscribes matching email addresses in Adobe Commerce. |
ddg_automation_platform_modified_contacts | Fetches recently-modified contacts in Dotdigital and resubscribes matching email addresses in Adobe Commerce. |
ddg_automation_pending_contact_checker | Checks Dotdigital to see if pending opt-ins (for abandoned carts and program enrolment) have been confirmed. |