How to design an experiment to test the effectiveness of SEA Brand?
Get started - quick and cheap
To begin testing marketing effectiveness, a marketing team should transform their most important hypothesis into experiments. It is advisable to start with inexpensive and quick experiments in order to learn rapidly. Each experiment will help reduce the risk of marketing teams investing time, energy and money into inefficient marketing activities with excessive budgets or no effectiveness at all.
Evidence
Every experiment has a specific runtime to generate sufficient evidence from which a team can learn. We ensure that experiments are conducted like scientists, ensuring that the evidence is clean and not misleading.
A good experiment
A good experiment should be precise enough for team members to replicate it and generate usable and comparable data.
• Defines the “who” precisely (test subject)
• Defines the “where” precisely (test context)
• Defines the “what” precisely (test elements)
Components of an experiment
A well-formed marketing effectiveness experiment is made up of four components:
- Hypothesis
- Experiment
- Metrics
- Criteria
Experiment: How much budget should a company allocate on SEA Brand?
It is important to note that our approach is inspired by Alex Osterwalter and his methodology "testing business ideas" from Strategyzer. We have adapted the test card specifically for marketing purposes, ensuring that our experiments are well-structured and aligned with industry best practices.Test Name: SEA Brand EffectivenessDuration: 12 weeksStep 1: Hypothesis
Our belief is that: SEA Brand protects our business by directing new users and existing customers to our service instead of our competitors.Step 2: Test
To verify this hypothesis: We will measure the impact of a 7.000.000 € SEA Brand budget per year on our new users and existing customers.Step 3: Metric
And measure: The total number of homepage hits, new sign-ups, and logins from search engines will be measured by 1) temporarily reducing and 2) temporarily pausing SEA Brand, each in four intervals.Step 4: Criteria
We will be correct if: The total number of homepage hits, new sign-ups, and logins are significantly influenced by the presence of SEA Brand.
Learn
By conducting this experiment, we aim to gather data that will help us evaluate the effectiveness of SEA Brand in driving user engagement and conversion. The 12-week duration will provide us with sufficient time to generate meaningful insights and make informed decisions regarding our marketing strategy.
Sneak Peek, before we delve deeper: We collaborated with a team that conducted a similar experiment and achieved a significant breakthrough. As a result, they were able to reduce the SEA Brand Budget by 80% (5.600.000 €). They implemented device modifier and day pacing, and reallocated 80% of the budget to campaigns aimed at acquiring new customers and generating additional revenue.
As Richard Feynman, an American theoretical physicist, famously said, "It doesn’t matter how beautiful your theory is, it doesn’t matter how smart you are. If it doesn’t agree with experiment, it’s wrong." Hence, we believe that conducting rigorous experiments is essential for validating marketing strategies and driving business success.
In the next blog post, i will discuss the technology behind our experiments, why web analytics can be challenging for these experiments, and explore the associated challenges. We will take a closer look at GDPR, Consent-Management, Python Scripts, measurement endpoints, PostgreSQL and reporting segmentation.