Establishing a Good Marketing Effectiveness Practice

But how do you ensure marketing effectiveness measurement is robust, defendable and clear? Here are some top tips for you to consider.

It has to start with objective alignment!

The first step is to be clear on your objectives: what is the campaign’s purpose? Is it to drive up knowledge of the brand? Or is it a reminder to purchase again? Are we trying to increase the reliance on online as its higher margin? By being clear about what you want the campaign to do, and to which audience, you are starting to define which metrics you should be measuring against, and therefore the definition of the KPI you should track against (for example, increasing awareness if your goal is to increase brand knowledge). This article by data guru Avinash Kaushik is worth a read on further defining the right KPIs.

Set realistic targets!

Once you’ve chosen the KPI, you will need to set realistic expectations of how you expect the campaign to move the KPI. For this, look to the past to see how much the metric has moved; if there hasn't been much variation, then perhaps you shouldn’t expect a huge increase. If there is a lot of movement, what looks realistic in terms of uplift? Make sure you consider seasonality. For example, look at the three-year pattern: are there certain times of the year that are always up? If so, take that into account.

How will you measure?

With targets set, you will need to think about how you are going to measure the campaign before you deploy it. Some techniques include:

• Random control tests (RCTs) and geo testing
• A/B tests
• Measuring against a historic baseline
• Multi Touch Attribution (MTA)
• Market Mix Modelling (MMM)

Knowing which technique you will use helps you define the shape of the campaign. Knowing which technique you will use helps you define the shape of the campaign. For example, if you undertake geo testing, you will need to identify the most appropriate geographical area for your activity to occur within and a comparable area to use as a control. Meanwhile, for MMM, you will need to ensure you have sufficient media spend levels and variation to enable you to get a read on the impact.

Make sure the test spend has the following key attributes:

• Is there enough spend to move your KPI?
• Is it shaped so you can get as clear a read as possible (i.e. bunch it up, don't spread it out)?
• Is it at a time that will conflate with other impacts?

Execute and measure.

Once you have the right objectives and metrics, know which measurement method you are going to use, and the campaign is successfully deployed, it is time to then execute and measure. When it comes to measurement, be conscious of limitations of your measurement technique.

RCTs, geo and A/B approaches are easy to deploy, simple to understand and can be deployed internally. However, there are some limitations to these techniques, which can prevent them from giving a full read on the effectiveness of your activity.

First, there's difficulty getting a read on the "carryover" of the campaign (often called the memory effect), which is the effect the campaign continues to have after the campaign has finished. These approaches also struggle to measure the impact of specific media activity onto other channels; for example, running social activity can boost PPC. These findings are key when trying to understand a full view on your media performance.

These methods are also unable to provide information on scaling the results. A test spend of £20k in one region will not have the same ROI as a £5M national campaign. Be aware of diminishing returns. You can get around this by upping the investment in increments and continuing to test.

MTA is great at giving detail, can be relatively easy to set up, and gives you a good relative read. It is not, however, an incremental analysis, so it is not reliable for ROI calculations.

MMM is incremental and includes a full read on all drivers of your KPI. However, it is blunt (you need to spend at least £100K on a campaign), does not give as granular detail as other techniques, and can be expensive.

Considerations for in-housing measurement.

So, as a client what could you do yourself?

• RCTs, geo and A/B tests: Most of the time, there’s no real need for external partners.
• Multi Touch Attribution: Give it a crack, it’s fairly straightforward and you can use techniques like Markov chains. But you can only use traditional MTA for a short while: with third-party cookies going away, there is a longer term need to invest in cookieless solutions.
• MMM: Great to in-house if you have the scale, but you need to keep a team busy and fulfilled. This, therefore, only works if you are either an enterprise or global company spending over £100m on media.

Be aware of a couple watch-outs for in-housing.

• Don’t use data scientists for MMM—use econometricians. We’ve seen time and time again that where data scientists have been tasked with building MMM models, it very very rarely works. Data scientists and econometricians see things in different dimensions.
• Econometricians are hard to find, and harder to keep hold of. You need to make sure the work is varied and interesting for them to stick around.
• Make sure there is enough work for at least four people. Otherwise, if you are reliant on one econometrician and they leave, it's very hard to get someone else in to pick things up.
• Career progression: A consultancy can always get bigger, but an in-house enterprise team hits a limit. So then you have no progression and people leave. Or you move them into non-econometric roles but then you still have the problem with recruiting in a niche field.
• Method stagnation: There is less opportunity from your econometricians to learn new techniques from larger teams working on other clients. So there is a danger that your capability starts to fall behind the curve—unless you have really high staff turnover, which you’ll need to have a big enough team to support.

However, if you are large enough and have the right team, then in-housing can save a lot of future money on consultants, whilst keeping complete control of your data and models. To keep it moving, and to make sure you don't stagnate, consider using an external partner to help refine your MMM approach.

Summary

In conclusion, getting the start nailed down is critical in marketing evaluation. Align objectives with KPIs and how you lay down the campaign. Think carefully about which technique you will use to measure, and whether you do that internally or with external help. In these uncertain economic times, there has never been a greater need to get this right. Bon chance!

For more information on how we can help measure drivers of growth for your business visit our Measurement page or contact us.