Issues displaying previews for Dynamic Content in email campaigns (All regions)
Incident Report for Dotdigital
Postmortem

Summary of impact:

At approximately 06:30 UTC on Wednesday 26th October 2022, after a platform update, campaign previews containing Dynamic content and our Spam check feature failed to work for some campaigns. From approximately 12:44 UTC, previews not containing Dynamic content began to fail too.

All issues were resolved by approximately 13:30 UTC by rolling back the responsible changes.

This issue only affected previews and Spam check, meaning no other features were affected and sends continued as normal.

 Root Cause:

We made a change to a low-level personalization component that wasn’t compatible with previewing Dynamic content and Spam checks. Test sends and full sends that included Dynamic content continued to work as expected, however, campaign previews failed to load correctly and would’ve appeared blank.

Mitigation:

The timeline (in UTC) for resolving this issue was:

  • 09:42: We identified the issue
  • 12:42: We released an initial fix for the issue
  • 12:44: We released a fix for an unrelated issue which unintentionally overwrote this fix, causing other previews to begin failing too
  • 13:30: We released a secondary fix to resolve the issue.

 

Next Steps:

To prevent this issue reoccurring, we’ll:

  • Extend our testing environments to allow for greater testing on our Spam check feature
  • Review our QA process to make sure robust testing is done on these features to prevent this happening again in future.
Posted Oct 27, 2022 - 12:03 BST

Resolved
Everything with previews is back on track now. We’ll write a detailed report on the issue we’ve experienced today. It’ll be posted on here as soon as it’s ready (in a day or two). Sorry about the disruption.
Posted Oct 26, 2022 - 13:53 BST
Identified
We've isolated the issue and we’re now taking care of the problem. Thanks for your patience, everyone. Regular service with previews will resume again very soon.
Posted Oct 26, 2022 - 11:53 BST
Investigating
We’re seeing an issue with displaying previews for Dynamic Content in email campaigns at the moment. We’re working on a fix as our highest priority. Sorry it’s happened, folks! We’ll have another update for you soon, stay tuned.
Posted Oct 26, 2022 - 10:58 BST