In 2020, the Beethoven pet store chain switched to a new CDP with the aim of enhancing its communication with customers and increasing the online store’s revenue. The switch to the new platform allowed trigger warnings to be set up, prevented revenue loss and unified and segmented the database. It also provided the opportunity to test a new neural network algorithm that forecasts the best sending […]
To all success stories
September 26, 2022
Beethoven’s Story: Using a Neural Network to Forecast Best Sending Time and Reactivating an Invalid Database
-
GoalIncrease online store revenue
-
SolutionSwitch to a new CDP
-
ResultTarget revenue was exceeded by 30% and reached the pre-CDP switch level
-
TeamLead Marketer at Beethoven, Mindbox Manager
-
Term1 year
-
ITERP, Mindbox marketing automation platform
-
Noteworthy featureThe Next Best Action algorithm helped determine the best sending date for emails, which then became more effective than trigger-based campaigns
Pet store chain. Founded in 1993
Andrey Lobur, Lead Marketer at Beethoven
In 2020, the Beethoven pet store chain switched to a new CDP with the aim of enhancing its communication with customers and increasing the online store’s revenue. The switch to the new platform allowed trigger warnings to be set up, prevented revenue loss and unified and segmented the database. It also provided the opportunity to test a new neural network algorithm that forecasts the best sending time for emails to increase repeat purchases. In this article we will be analyzing:
- What triggered the decision to switch CDP;
- How the neural network’s LSTM model (the Next Best Action algorithm) increased the mailing quality, making them more efficient than trigger campaigns and helped increase mailing revenue by 8.5 times;
- How to segment the database, optimize bulk campaigns, and what to do with invalid email addresses;
- Future plans for improving the channel.
Results after Switching to the New CDP
Target revenue was exceeded by
30 %
and reached the same level as before the implementation
The CDP now contains
2.5 times
more valid customers thanks to the unified database and data cleansing
Revenue from emails increased by
8.5 times
with the Next Best Action algorithm (based on last-click attribution in A/B tests)
The brand now sends
23 times
more targeted emails (compared to trigger emails), aimed at boosting repeat purchases with the Next Best Action algorithm
Why Did Beethoven Decide to Switch to Mindbox
opinion
At some point, I realized that if we want to grow, we need better partners, so we decided to switch to Mindbox. We’re launching the app soon, so I’m interested in working with recommendations for the app, and sending push notifications and triggers, among other features. It’s much easier to set it all up in Mindbox, and I’m sure we’ll be able to sort it all out simply using the platform.
So far, we have moved standard trigger chains like “Abandoned Cart” and “Abandoned Views,” taken a look at how they work and set up triggers, time intervals and a priority sequence. I am grateful to the Mindbox manager who supervised these settings. The only thing that remains for us is to see how the recommendations work.
Andrey Lobur, Lead Marketer of Beethoven
| Before the switch to Mindbox | After the switch to Mindbox | |
| Workflows | There was no understanding of how workflows and algorithms worked | Transparent workflows that can be changed and customized according to the company’s needs |
| Segmentation | Pre-configured segmentation conditions, manual list uploads | The conditions and attributes for segmentation are set by the company; the platform runs segmentation and sends campaigns |
| Recommendations | Pre-configured algorithms with recommendations meant that it was impossible to analyze their operation or change the settings manually | Selecting products and recommendation algorithms is now possible. For example, if the email conversion is low, it’s possible to change the product and display only discounted or new products |
| Trigger Chain Management | The system didn’t track whether emails from different trigger chains were duplicated and still sent a similar email | If an email is sent to a customer and is duplicated in another trigger chain, a similar email won’t be sent |
| Attributions | It was impossible to track the true effectiveness of the mailing because the system only counted clicks in the email itself | The platform attributes the customer’s purchases to the mailing list if the customer has made no other actions for 30 days. For example, if a customer received a mailing, and a couple of days later purchased the product directly through the website, the platform will attribute the purchase to the mailing |
How the Neural Network Helped Forecast the Best Sending Time for an Email with a Repeat Purchase Offer
Pet owners order products such as pet food and litter with a certain frequency, as they have to replenish them from time to time once they’ve run out. The task is to predict when the customers will need the product next and to send them a correspondening email with an offer.
The Next Best Action neural network algorithm helped forecast the probable purchase date for each customer. The neural network analyzes the interval between the customer’s purchases and how often the customer opens emails and makes purchases after receiving them.Then it suggests the Best Sending Time for an email with recommendations. The email lists the products that the customer purchased previously and contains a “Repeat My Previous Order” button. After clicking the button, the customer is redirected to a page with a cart with the same preselected products.
We conducted an A/B test to evaluate the results of the algorithm. Some of the customers received emails based on the average time between purchases; others received emails based on the neural network’s forecast. The significance level of the A/B test results is 95%. The NBA algorithm produced higher results compared to the trigger, which sets a fixed time between email sendings:
| Indicator | Increase |
| Targeted emails | 23 times in absolute value |
| Open rate | 17 times in absolute value |
| Revenue for email sending | 1.2 times in absolute value |
| Conversion rate | 1.5 times as a percentage |
In February, the NBA algorithm was launched for the entire database and, in just 2 weeks, it overtook all triggers in terms of efficiency and reached 26.9% of the total revenue received from automated campaigns. The only original trigger that remained competitive with the NBA algorithm was the one that sends an email with a promo code 15 days after a customer’s purchase. If the positive results we see from using the NBA algorithm persist, we will consider disabling the trigger with a promo code for a future order and replace it with an algorithm. This step will increase our margins without sacrificing any customer benefits. The customer will still receive reminders about orders.
An email with a “Repeat My Previous Order” button is sent when we predict the customer is about to run out of their previously ordered goods. The Best Sending Time is calculated by the NBA neural network algorithm.
How the Segmentation Was Updated
Previously, only 30% of the entire customer base was segmented, and the database did not include any data from ERP software used by Beethoven. Moreover, in order to create a segmented campaign, an email list had to be requested from the service manager, and the mailing had to be configured manually from an Excel spreadsheet.
We’ve now collected data on the existing five pet segments (cats, dogs, birds, small pets, and fish), configured the correlations between orders and views and covered 100% of the database. Now we know the interests of our customers for certain and can interact with them in a targeted manner.
To engage the target audience more accurately, trigger campaigns for new products are also divided into categories: cats, dogs, cats and dogs, birds, small pets, fish, and the pet pharmacy. An email with new products is sent if the customer has not made any purchases within the last month. The segment is recalculated every day and includes new products that have not been included in campaigns over the last 30 days. A trigger checking the availability of the products in the segment and whether the customer received an email with them the previous week is launched weekly on Thursdays.
The campaigns for the “Cats” and “Dogs” segments are sent before the “Cats and Dogs” segment, in order to avoid email duplication for those who have several pets.
To reactivate customers who have not made orders for a long time, in addition to new products, we also offer a discount.
How Bulk Campaigns Were Optimized
Now, bulk emails are automated and sent only twice a month. This saves marketing resources and keeps only the customers who want emails in the database. In addition, infrequent emails to inactive customers positively affect the spam profile of the store’s domain.
We only send out promotional campaigns (e.g., product discounts). The company automatically generates emails from a template banner in which the sizes of discounts and featured products and brands are inserted. Then, the email is uploaded to the CDP. The marketer changes the templates every few months to diversify the mailing.
The mailing list goes either through the entire database at once, or by segments for owners of cats or dogs. Because mass emails are rarely sent and trigger emails are sent only to those who visited the website, the company is able to avoid spamming customers. In addition, it helps gently remind churned customers about the company.
Bulk campaigns are sent by segments depending on the customer’s pet
Trigger campaigns overtook bulk campaigns in December 2020, which signified the growth of the active customer base. The channel grew due to the new CDP as well as due to additional advertising and affiliate programs.
How Were Customers with Invalid Email Addresses Reactivated
The customer’s email address can be entered in the list of invalid email addresses if the platform detects an incorrect email address (for instance, gmail.net instead of gmail.com), or if the mailing fails to deliver for any other reason. This measure is necessary to understand the real size of the customer base available for communication.
Some customers with invalid addresses can be activated. The email may be valid and belong to a real customer that had a full inbox at the time of receiving the email, or the server may have rejected the email for other reasons. To give these contacts a second chance, an email is sent once a month only to invalid addresses. Some of the emails were never delivered, hence, these customers were excluded from the campaigns. However, most of them were actually delivered. The open rate for delivered emails averaged 66%; the click rate was 14%, and some emails even led to purchases.
At the same time, it is crucial not to send more than one email a month to invalid customer emails and exclude email addresses that are clearly invalid, such as those with non-existent domain names. Otherwise, it may have a negative effect on the spam status of the sender’s domain.
How Do We Plan to Develop the Digital Channel
- Increase revenue from campaigns by 15% in 2021;
- Launch end-to-end recommendations on the website and in the app;
- Launch AMP mailing lists;
- Switch service emails to the platform.