When it involves having a powerful testing mentality on the cornerstone of your progress advertising technique, no one does it higher than Sue Cho, Head of Lifecycle Marketing at Calm. She’s been an Iterable MVP for years, ever since she served because the Director of Email Marketing at Autolist.
We just lately sat down with Sue to recap her latest webinar with us and Amplitude on the three truths she discovered by way of analytics and testing.
Read on to be taught the ins-and-outs of Calm’s tech stack, their most vital metric and extra invaluable insights!
An Interview With Sue Cho From Calm
Tell us about your self, Sue.
I’m the Head of Lifecycle Marketing for Calm and I oversee all issues retention, engagement and conversion. I run campaigns on channels, like email, push notifications, and even junk mail (which we name snail mail).
I’ve been doing email and retention advertising for 10+ years, largely targeted on the subscription e-commerce house. I’m additionally an energetic member of Email Geeks, which is a worldwide group, and I run the San Francisco chapter of the Email Geeks meetup. I’m additionally a constitution member of a corporation known as Women of Email.
I’ve been with Calm for about two years and in that point, I’ve been in a position to attract some actually cool insights due to all of the applied sciences that now we have in place.
In your individual phrases, what’s Calm?
Calm is the #1 app for sleep, meditation and relaxation. We have been named App of the Year by Apple in 2017. We have content material for sleep and meditations, and now we have unique music, white noise, soothing backgrounds, grasp courses, and even physique workout routines. We have 80 million international downloads.
My favourite portion of the app is definitely the enjoyable sleep tales, like “Once Upon a Time in GDPR,” the place a stunning, boring British male voice reads you the whole regulation e-book of GDPR. I’m undecided if that’ll provide you with nightmares or get you to a soothing slumber, however now we have it.
We even have sleep tales from folks like Matthew McConaughey, if you need him to learn you a bedtime story. And now we have some unique Calm music by some fairly cool artists, like Sam Smith and Lindsey Stirling.
What’s the tech stack at Calm seem like?
Our progress advertising device is Iterable. I’ve truly been an Iterable shopper throughout a number of totally different corporations. I’m a giant fan. It’s one of the vital highly effective automation instruments for email and push notifications there may be out there.
Then we use Amplitude as our information aggregator and visualization device. The cool factor about these two instruments is that we even have a bi-directional integration the place Iterable auto-streams information into Amplitude about email opens and ship information, and all the consumer properties nested in these, and now we have the potential to push Amplitude cohorts and segments into Iterable as nicely.
How are you utilizing this information?
With all of this information—together with from Iterable—feeding into Amplitude, it permits us to visualise full-funnel on what a consumer does.
We can have a look at issues like:
- User conduct from an app obtain
- Whether or not they open an email
- When they join a subscription
- What they really do as soon as they open the app
In different phrases, these applied sciences mixed collectively permit us to see past superficial metrics, like opens and clicks, and provides us visibility into what the consumer truly is doing within the app after they open and click on.
Why do you’re keen on analytics and testing?
I believe it’s actually thrilling and gratifying to see the nice outcomes that you just’ve had from working exams. Whether or not the check outcomes are what you predicted, you will get validation by way of concrete information, which helps you make goal and knowledgeable selections.
And by optimizing our applications to provide the customers what they need, we’re in a position to get optimistic suggestions, like a latest Facebook remark we obtained that mentioned, “The best part of Sunday night is when the Calm email comes, and the preview for the upcoming Daily Calms, and what you need to work on, is there.”
I’m so used to being at corporations and getting emails from indignant customers saying, “Stop spamming me. Get me off your list. This is junk.” And now we get folks writing in upset that they didn’t get the Sunday email that talks concerning the Daily Calms, so it’s actually gratifying.
What’s an fascinating perception you’ve realized from testing?
We discovered by way of information that the “Sunday Scaries” are actual. We noticed a definite cyclical sample of utilization: People begin to freak out on Sundays as a result of they’re stressing out over all of the work that’s coming on Monday. Clearly, there’s a sample the place each Sunday, we get an uptick. On the weekends, folks don’t want to achieve for Calm anymore.
But what about all through the day? Instead of each day charges over 60 days, we checked out hourly charges over 5 days. Now we noticed a double hump cycle: The bigger hump is at nighttime round 10 PM when persons are attempting to go to sleep, and the smaller hump is within the daytime when persons are preparing and getting their thoughts straight for the day. So we optimized most of our campaigns to exit early evenings, early mornings, Sundays, and by no means, ever, ever on a Friday or a Saturday.
What is your most beneficial metric, that you just have a look at each day, weekly, or month-to-month?
#1 is at all times income. And then now we have the supplementary metrics that result in income that we additionally focus on: The variety of coupon redemptions and conversion metrics, like install-to-trial, trial-to-paid, the variety of distinct days interacted with the app throughout a trial, and the variety of periods accomplished whereas in a paid membership.
How do you measure conversion?
We have a conversion occasion for net and even have webhooks to fetch conversion information from Apple and Google. To entry these conversion occasions and attribute them to campaigns, I’ve a number of choices due to our tech stack:
- In Iterable, I’m in a position to outline a customized conversion occasion for each marketing campaign. I can set the attribution interval primarily based on an email ship, open or click on, in addition to the time interval between the email occasion and the conversion occasion.
- Amplitude additionally receives a stream of Iterable occasion information. I can use Amplitude’s characteristic “Funnel Analysis,” which permits me to see what number of customers did X occasion after Y occasion— i.e. what number of customers accomplished a session after opening the email? Amplitude additionally means that you can set a customized attribution interval.
- We even have UTM / Google Tags, which we don’t leverage as a lot.
Do you could have a standardized framework for a way lengthy you run your exams?
Yes and no. The reply for a way lengthy we must always run the check is at all times dependent on once we attain significance. For advanced exams, we pull in a knowledge scientist to research our common visitors and inform us—earlier than the exams begins—how lengthy we have to run it with a view to achieve significance.
For smaller exams like topic strains, Amplitude has an A/B testing significance calculator. So I, truthfully, simply refresh it till it reaches significance. But we will at all times pull in a knowledge scientist, and they’ll calculate how a lot energy we’d like and how a lot time we have to get to the facility stage.
How do you prioritize what to check?
Money [laughs]. We have a look at the largest alternative, and it’s often a operate of quantity. With that mentioned, if administration desires to check one thing, typically that simply trumps cash.
To recap your webinar, what are the three truths you realized by way of analytics and testing?
- Don’t be fooled by superficial metrics, particularly when working with email. A lot of individuals get distracted by opens and clicks, however often, our finish aim is a conversion or a sure kind of exercise. Make certain your campaigns are optimized for the precise occasions that you really want the consumer to do.
- Learn out of your most engaged customers. If you don’t know the place to begin, research the people who find themselves utilizing the product probably the most. That’ll nudge you in the precise route on what content material to indicate and how typically it’s best to message. Build your marketing campaign together with your famous person consumer in thoughts to persuade the others that aren’t as engaged to begin exhibiting and training these behaviors.
- Neither #1 nor #2 are potential except you put money into the instruments that can enable you achieve these insights quicker. Empowering your workforce to have instruments to have the ability to draw their very own insights and iterate shortly, is essential.
To be taught extra about Sue Cho and Calm’s progress advertising and testing methods, watch our webinar on-demand.