How to run growth experiments

How to run growth experiments

Running growth experiments isn’t rocket science. However, many founders and startups overlook the importance of structuring their experiments and tests and get too excited by the test itself. When you are testing something within your business, learning is paramount. Everything you can learn will further guide your business towards success. When you have qualified a new idea for your business, the following growth experiments template is an excellent way to lay it out so that you and everyone on your team knows what the test is, why you are doing it, and whether it worked or not.

 

The 7 steps of an experimentation framework are:

  1. Overview
  2. Hypothesis
  3. Control
  4. Test
  5. Measurement
  6. Results
  7. Next Steps

 

Overview

This is a brief synopsis of what the problem is or the area you are trying to improve.

Example: No one is opening our emails so no one is seeing our amazing deals.

Hypothesis

Every test should start with a testable hypothesis. For example:

By [doing] or [not doing] [something – an action] we believe that [a metric] can be impacted.

Example: By changing the time we send our emails, we believe we can increase email open rates.

Control

A control is what we are measuring against. Usually historical data, it serves as your baseline.

Example: Over the past month, our email open rate has been 12% when we send at 8am.

Test

This is the entity that you are changing, adding, or removing which is measured against your control. You may have more than one variant that you would like to run either simultaneously or sequentially.

Example: We are going to test sending emails at 2pm and 6pm.

Measurement

Determine how you will measure the results and over what time period. You should establish how to measure before starting anything. Make sure you know what’s happening. Also make sure you stay consistent in what you are measuring by using the same set of data from the same platform throughout. Consistency is key in ensuring you have results that you trust.

Example: We will measure the email open rates using our email provider for the next month – we will finalize the results a week after the last email of the month is sent.

Results

What did we learn with each test or variant?

Track detailed notes and metrics on a daily or weekly basis. Testing doesn’t happen in a silo so be sure to make note of any external or internal affects which may have had an affect on the results.

Example: Variant B was over 100% more successful each week than our baseline so our test was a success. Variant A and B both outperformed the baseline across the month with Variant B being the most successful – it appears the later we send the email, the more likely it is to open. Other notes included a successful paid acquisition campaign so we had a lot more qualified emails come in for Week 4 which may explain it being the strongest week.

Next steps

Will you run another iteration?

Is this a test to kill?

Will you try something else?

Does the test become a roadmap item?

Example: We will update emails sent to later times and kill the time based element of the test. Given the large increase in performance, we will also start testing what days we send email for a new item.

Learn more about running great experiments

We have a whole set of growth models in our Velocity-Growth-Mastering-Growth-Tools spreadsheet.

If you want to dive a lot deeper, I’d urge you to watch our Mastering Growth program. We used to sell this for $1,000 per seat, but it’s now free 🙂 

Share the Post:

Related Posts