Creating a loyalty club: how the right KPIs can improve data quality

2 Aug ‘17

Loyalty clubs are in a league of their own in marketing. They strive to unite the target audience around common ideas, values and interests, turning them into brand advocates. An FMCG company contacted Mindbox for marketing consulting regarding how to create a strong loyalty club.

The project team understood that the first impression is the most important part of attracting the audience, so they decided to invite people to join the loyalty club personally with the help of consultants on the streets of Russia’s cities. The consultants talked about the advantages of joining the club and used a special tablet app to input contact information into the application forms. The information was automatically transmitted to Mindbox. From there, SMS and email sequences were used to get customers to join the loyalty club.

While everything on the service side worked as well as could be expected, the human factor produced systematic errors: a lot of application forms were filled out, but the quality was low. They just couldn’t seem to get enough members to join. The project slowly evolved from building a loyalty club to creating a «hardworking consultant club» to provide quality data.

Strength test

We analysed the application forms after the consultants had been working for several months. Here’s what we found:

  • the consultants filled in the phone number in 100% of the application forms: this was mandatory
  • 15% of the application forms had the email address filled in: it’s harder to write down an email address than a phone number when someone tells it to you, since it consists of letters
  • 40% of the phone numbers and 60% of the email addresses weren’t valid: the messages sent to them weren’t delivered
  • the application forms showed signs of fraud: for example, some emails were city names like or, and the most popular phone number doesn’t exist: +7-999-999-9999

Our analysis showed obvious mistakes in the project implementation. We decided to develop KPIs to improve application form quality, thereby increasing loyalty club membership.

Application form quality requirements

Comparing the results from SMS messages and emails showed that email had a higher conversion rate and lower cost, meaning that the email field in the application form was more important than the phone number.

In both channels, some messages weren’t delivered. In general, there were three problems:

  • due to either cheating or mistakes, some consultants were inputting non-existent phone numbers or email addresses
  • some customers didn’t want to give their real contact information for some reason
  • sometimes there were problems on the receiving side: the subscriber might have been offline or turned off their phone temporarily, their inbox might have been full, or the message might have wound up in their spam folder

With email, we figured out how to determine the root cause of the problem by requesting detailed status messages from the mail servers.

It was harder with phone numbers, but our analysis helped. It turned out that only 5% of messages were delivered by sending the SMS again a couple days after the first attempt. This implied that 95% of the time, the phone number was wrong.

We also didn’t want to punish the consultants when customers wouldn’t give their real contact info, so we accounted for this when determining the KPIs.

Choosing KPIs

The most important thing for us was getting valid contact info. The email address was most valuable. As a result, we wound up with two KPIs:

  1. the share of application forms where at least one of the contacts was valid
  2. the share of application forms with a valid email address

The first KPI is necessary for end-to-end quality control of the application forms. The second KPI helped to control the quantity and quality of email addresses in the application forms. Importantly, by increasing the share of application forms with a valid email address, the consultants were automatically increasing the share of application forms where at least one of the contacts was valid. This helped to motivate the staff.

KPI targets

Our priority was to make sure that the KPI targets were feasible, so we began gradually. Our goal for the first stage was to eliminate cheating, but we understood that the consultant isn’t always at fault for incorrect phone numbers and email addresses: sometimes the customers are responsible.

We calculated the average performance level of the new KPIs and set this as the first target. This did not hurt those who had been collecting applications honestly before, but forced unscrupulous consultants to change their approach. As a result, the average performance improved.

We then gradually made the KPIs stricter, setting the target at the average level of the top 20% of consultants. We moved gradually, calmly and without applying psychological pressure on the staff.

Switching to KPIs

The process of switching to a KPI-based pay structure is far from simple. People don’t like to change the way they work. Transparency and consistency helped to smooth out the process.

We did several things to make the transition easier:

  • we showed consultants what they were doing wrong ahead of time, so that they could correct how they work before it began to impact their pay
  • we came up with automated reports for consultants’ comments, so that it would be easy to work with them
  • we changed the script that the consultants used when talking with customers and reduced the number of fields on the application forms, then moved the contact information to the top of the application forms so that customers would fill that out first

These measures made for a smooth transition to the KPIs. 

Sample Excel report for KPI quality control


The project has been underway for two years now and has been under continuous development. We’ve modified the KPIs a couple times and changed the target thresholds. But the most important results are the KPI improvements:

  • the share of application forms with a valid email address has risen from 6% to 26% (both the share of forms with the email address filled out and their quality have improved)
  • the share of application forms with a valid phone number has grown from 60% to 80%
  • the improved quality has led to a 60% increase in the number of application forms converted into loyalty programme members

The loyalty club project was successful despite numerous difficulties. FMCG is a sector where the result justifies the investment. But marketing technology and services are not useful without quality staff.

Yulia Turkina, Lead Analyst at Mindbox

Tell us a little about yourself

We’ll respond within 24 hours

Partnership request

Typically we’re answering within 24 hours