How to Measure: Email Marketing

Email marketing is a great way to connect with your audience, so much so that 77% of marketers saw an increase in email engagement in 2023. Whether it’s with a retargeting email or a reminder about an upcoming sale, email is a medium that should not be overlooked. So, with that, how do we go about measuring email as a channel correctly? 

In short, the impact of email marketing can be measured with MTA, MMM, and incrementality testing, or ideally, a combination thereof (triangulation method). However, let’s have a closer look… 

Multi-Touch Attribution (MTA)

MTA measures the impact of email marketing using user-level click tracking. This also means that MTA provides the impact of email clicks, if no one clicks on the email, MTA will not estimate the impact. Yet there could still be one, if users google for the brand after opening the email, for example. While this may lead to a potential underestimation, in practice, this is probably rather small, since users, after all, still click on email links. Two other effects, in turn, lead to a potential overestimation. 

Firstly, trackability is usually still quite good. This is because deterministic user identification relies on hashed email addresses, at least in the Adtriba case, meaning, you have user IDs for users with whom you have emails. Naturally, you need to have somebody’s email address to send them an email. So, by design, we are much more likely to “stitch” an email click touchpoint into a user journey than, for example, a social touchpoint. 

Secondly, there is selection bias. Users who provide their email address are usually more engaged and thus intrinsically more likely to buy or have already bought, and that’s why we have the email address.

Overall, MTA can still provide a rough estimate of the email impact. Importantly, though, the impact can be assessed at a very granular, individual email level.

Marketing Mix Modeling (MMM)

MMM, on the other hand, delivers a broader view of how email marketing works in tandem with other marketing channels. In measuring the impact of email marketing, MMM can provide insights into how this channel contributes to overall sales or brand awareness. By considering various factors such as historical data, seasonality, and competitive actions, MMM helps marketers to not only gauge the performance of email marketing but also to understand its role within the larger marketing mix. This can guide strategic planning for marketing budget allocation, enabling the balancing of investments across different channels for maximum overall performance. Again, we have to watch out to circumvent a few biases here. 

First of all, we should take a look at which types of email campaigns we want to include in the marketing mix model. Emails resulting from automation flows that get triggered at different points in a user’s lifecycle, e.g. "Welcome,"  “Please come back,"  and “We haven’t heard from you in a while,"  provide probably more random background noise than useful variation. Those should be left for MTA (or testing) and excluded from the MMM. Push campaigns, e.g., Easter campaign, Christmas campaign, new collection campaign, however, should be included. 

Secondly, which KPIs do we include? While ideally we often want to include spend in an MMM, for email it may not always be the ideal choice, e.g., if there is a monthly fee for the email provider, or the pricing is based on different tiers and bulk packages. However, we also don’t want to use opens or clicks since this is already an outcome of the campaign. Therefore, at Adtriba, we usually recommend send-outs to quantify email marketing in MMM.

Incrementality Testing

It is always a good idea to incorporate incrementality testing into any measurement setup, to get a better understanding of the “true” causal impact of a marketing tactic. This, in turn, can be used to further calibrate and optimize the modeling approaches explained above. 

Email campaigns are uniquely well positioned for testing, since “true” experiments, RCTs, and randomized controlled trials are still possible. After all, the only thing we need to do is randomly separate the email address base into two subsets, where one subset receives the email in question and the other doesn’t. Provided the subsets are large enough and the effect of the emails is sufficiently large, we can estimate the true causal impact of email marketing. 

Randomized Control Testing (RCT), or A/B testing, also offers a more micro-level understanding of email marketing impact. The implementation of RCT in email marketing allows marketers to pinpoint specific campaign elements that resonate most effectively with the target audience. By continuously testing different variations of email components such as subject lines, content, layout, or call-to-action placements, RCT provides direct, actionable insights into what works best. These insights can be incorporated into future email campaigns, enhancing their effectiveness and, in turn, boosting the return on investment for email marketing activities.

Summary

In summary, MTA, MMM, and testing/RCTs are valuable tools for measuring email marketing's impact, each providing unique insights that can optimize marketing performance at different levels. Leveraging these tools together and combining their results allows marketers to comprehensively assess and enhance their email marketing strategies, driving stronger engagement, conversions, and business growth.