Measure conversion uplift via control group analysis

Modified on Thu, 07 Dec 2023 at 01:17 AM

NotifyVisitors Control Group feature is a robust analytical tool that empowers businesses to assess their marketing campaign effectiveness. It does so by comparing the performance of users who received campaigns with those who didn't, yielding critical insights into conversion rates and revenue generation. 

The feature comprises two distinct control group types: 

System Control Group, which provides a default exclusion rate for all campaigns

Campaign Control Group, allowing customizable exclusion percentages for individual campaigns. 

This versatility ensures consistent benchmarking and tailored campaign analysis, contributing to more informed marketing decisions. 

In this guide, we'll explore the intricacies of this tool and how it enables businesses to make data-driven decisions for optimized marketing.

System control group

The System Control Group provides a straightforward process that allows you to set a default exclusion percentage for users who will not receive your campaigns. This default exclusion rate remains consistent for all campaigns, providing a stable benchmark for comparison.     

To set the System Control Group, follow these steps:

  • Navigate to the "Settings" menu.
  • Access the "Campaign Control" setting
  • Now click on the "Control Group" section.
  • Click on the "create” option 
  • Now input a percentage value from 1 to 100 as per your requirement.

Once you've configured the System Control Group, it becomes an integral part of your campaign management process. Every time you send a campaign to users, you have the option to enable the System Control Group in the campaign editor. This ensures that the specified percentage of users, as set in the default exclusion rate, will be automatically excluded from receiving the campaign.

The consistent exclusion rate will serve as a cornerstone for measuring the impact of all your campaigns. It creates a baseline for comparison and allows you to evaluate the true influence of your marketing strategies on both conversion rates and revenue generation. It ensures that you have a stable reference point from which to assess the success of your campaigns across various scenarios, making it an indispensable tool for data-driven decision-making in your marketing endeavors.

Campaign control group

The Campaign Control Group is a strategic feature which provides more flexibility by allowing you to define different exclusion percentages for individual campaigns. Similar to the System Control Group, you can enable the Campaign Control Group in the campaign editor itself, and then in the percentage box specify the percentage of users to be excluded from that particular campaign. This dynamic group empowers you to customize the exclusion percentage on a campaign-by-campaign basis.

By seamlessly integrating the Campaign Control Group while sending a campaign, you gain the ability to fine-tune your marketing strategy with precision. It ensures that you can adapt exclusion rates to suit the specific objectives and target audience of each campaign. 

Let us understand the difference with an example-

If you've configured a 10% exclusion rate in the System Control Group and have been excluding these 10% of users in most of your campaigns, but for a particular campaign, you wish to exclude 20% of users, the Campaign Control Group allows you to do so without affecting the default system value and all other campaigns. This level of flexibility is essential for tailoring your marketing efforts to different campaign goals and target audiences.

Control group analytics

Control Group analytics provides a comprehensive understanding of your marketing campaign's impact. These analytics give you such valuable insights that serve as a basis to compare conversion rates and revenue between users who received your campaigns and those who didn't.

Conversion section

To analyze the impact of your marketing campaigns on conversion rates, follow these steps:

  1. Navigate to the "Campaign" section.
  2. Select the analytics of a specific campaign (available for email, SMS, and WhatsApp campaigns)and click on the email stats tab.
  3. Focus on the Conversion Section, which provides the following insights:
  • Firstly, it shows the percentage boost in conversions, representing the increase in conversions due to your campaign.
  • Further analysis breaks down the conversion rates for two user groups - those who received the campaign (target group) and those who did not (control group).

By studying these conversion rates, you can gauge the effectiveness of your campaign.

For example, the above image depicts 50% of users achieved conversion within the group that received the email campaign, in contrast to 28.57% who converted from those who did not receive the email campaign leading to a notable 75% boost in conversion due to campaigns.

Revenue section

To assess the financial impact of your marketing campaigns, particularly the revenue generated, follow these steps:

  1. Navigate to the "Campaign" section.
  2. Select the analytics of a specific campaign.
  3. Click on the email stats tab.
  4.  Pay attention to the Revenue Section, which offers the following insights:
  • It shows the total revenue boost and the percentage boost in revenue, which reflect the financial impact of your campaign.
  • Further it reflects the insights into the total revenue and average revenue per user (ARPU) for both user groups.

Total boost in revenue is calculated using the formula: [(ARPU in target group – ARPU in control group) × total target group users]. For example, if your email campaign resulted in an ARPU of 50 in the target group and 28.57 in the control group, the total boost in revenue can be calculated as [(50 – 28.57) × 220 = 4714.60]. This figure illustrates the additional revenue generated by your campaign.

The percentage boost in revenue is determined by the formula: {[(ARPU in target group – ARPU in control group) ÷ ARPU in control group)] × 100}. In the example, the boost percentage is calculated as [(50 – 28.57) ÷ 28.57 × 100 = 75%], indicating a 75% relative increase in revenue due to your campaign.

ARPU is a key metric calculated by dividing the total revenue by the total number of users. For the target group, ARPU is calculated as [11000 (total revenue generated from target users) ÷ 220 (total target users) = 50]. In the case of the System Control Group, ARPU is calculated as [2000 (total revenue generated from System Control Group users) ÷ 70 (total System Control Group users) = 28.57]. This metric allows you to understand the average financial contribution of each user segment.


In a competitive business environment, staying ahead of the curve is essential. The Control Group feature is a game-changer in the world of marketing analytics. By comparing the performance of users who received campaigns with those who did not, you can fine-tune your marketing efforts, increase conversion rates, and boost revenue generation.

Was this article helpful?

That’s Great!

Thank you for your feedback

Sorry! We couldn't be helpful

Thank you for your feedback

Let us know how can we improve this article!

Select atleast one of the reasons
CAPTCHA verification is required.

Feedback sent

We appreciate your effort and will try to fix the article