As a marketer, you want to improve your website’s performance and get more conversions. In many cases, A/B testing could be the answer.

By comparing two versions of a web page, you can determine which version achieves your goals more effectively.

In this article, we’ll walk you through the steps to set up A/B testing with Microsoft Clarity and GA4.

Microsoft Clarity is a free heat map analysis tool that has all the necessary segmentation features to set up A/B testing, especially when Google Optimize drops out as a marketer; You need alternative ways to run your tests.

The easiest and most straightforward way to set up A/B testing is to set up two versions of the website and direct traffic to each.

Using a Microsoft Clarity URL filter, you can segment and analyze data for different versions of your web page.

But what if you want to test different layouts of the page in live traffic without different URLs?

Luckily, Clarity has custom tags (and custom GA4 dimensions) so you can tag your users and filter them in reports.

What are Microsoft Clarity Custom Tags?

Clarity’s custom tags are any alphanumeric custom labels that you can assign to the visitor and later use to segment data and analyze recordings and heatmaps for different test groups.

Are there any limitations on custom tags in Microsoft Clarity?

There are no borders. You can add as many tags as you want to your project with no limitations or restrictions.

How to tag a visitor with Microsoft Clarity

Tagging is as simple as running a small piece of JavaScript code:

clarity("set", "experiment", "group_name");

But I want to guide you with a concrete, real-life example of how to use this from our experience.

At SEJ, we run various tests on different types of ads and webpage layouts to gain insights into how user behavior is influenced by factors such as banner ad type or webpage layout.

Examples of A/B tests we run:

• Static banner ads vs. animated banner ads.
• Left sidebar vs. right sidebar.
• Change menu labels.

The goal is to understand when users scroll deeper into the article and engage in reading—or whether changing menu labels can help generate more clicks.

1. A/B testing of static banner ads vs. animated banner ads

We use Google Ad Manager to load ads on our website and can therefore pass key values ​​to our ad server using the Google Publisher Tag API.

We distribute the traffic evenly using the Math.random() function in JavaScript, which returns 1 or 2.

To start the experiment, copy and paste the following.

We use the “ads_type” key with the predefined values ​​”static_ads” and “animated_ads” in GAM to also be able to report on ads on the GAM side, such as the CTR for each group.

Then copy in section of your website and paste the JS code. Alternatively, you can use the custom GTM HTML tag on every pageview where you serve ads.

<script>
   window.group_name = "animated_ads";
   let randomNumber = Math.floor(Math.random() * 2) + 1; // either 1 or 2
   if( randomNumber == 2 ){
      group_name = "static_ads";
   }   
document.addEventListener('DOMContentLoaded', function() {   
   //make sure publisher tag has loaded   
   if( typeof googletag != 'undefined' ){   
      googletag.pubads().setTargeting("ads_type", group_name );
   }
   //check if clarity has loaded and set tag "experiment" with values "static_ads" or "animated_ads"
   if( typeof window.clarity != 'undefined' ){
      window.clarity("set", "experiment", window.group_name );
   }
});
</script>

Normally, when the DOMContentLoaded event fires, the publisher tag and Clarity are loaded. If not, you can consider wrapping JS in a setTimeout() function with a small delay.

With the “ads_type” key in GAM it is possible to configure different types of banners that are displayed to each group. Because we set up this key as a tag value for the “Experiment” key in Clarity, we can analyze data for each group and create your reports.

If you need a specific setup that requires advanced coding, you can try using ChatGPT to write some code for you.

If you want to track how users’ conversion rates are changing in GA4, you can add a custom dimension in GA4 with the key “experiment” and populate it when the configuration tag loads by using the datalayer.push use.

dataLayer.push({ 'experiment': group_name });

Alternatively, you can use GTM JavaScript variable to retrieve the global variable value window.group_name that we set as a test parameter above.

And set up a custom dimension in configuration tag to pass this variable value as shown below:

Fill in the custom dimension “experiment” from the global JS variable window.group_name and voilà!

Now your test custom dimension is passed to GA4 and you can filter reports using the Experiment custom dimension.

(Quick tip: when naming your custom dimensions, make sure you don’t use one of the reserved parameter names so they work properly.)

2. Left sidebar vs. right sidebar

The principle is the same. Use the Math.random() function in JavaScript to break up the test.

<style>
/*when adding this class to the content div it swaps sidebar position */
.main_content_right{
flex-direction: row-reverse;
}
</style>
<script>
   // since we have no any css under .main_content_left class it will do nothing i.e. sidebar will be the default right;   
   window.group_name = "main_content_left" 
   let randomNumber = Math.floor(Math.random() * 2) + 1; // either 1 or 2. 
   //let randomNumber = Math.floor(Math.random() * 5) + 1; // random number from 1-5. use this if you want to run test on the sample of your traffic e.g. on the 25%.
   if( randomNumber == 2 ){
      group_name = "main_content_right" // we will use group_name as a class name and a custom tag at the same time;
   }
//If DOMContentLoaded has loaded run the code, otherwise attach an event listener   
if (document.readyState === 'complete') {
     move_sidebar( group_name )      
   } else {
   // DOMContentLoaded event has not yet fired
   document.addEventListener('DOMContentLoaded', function() {
       move_sidebar( group_name );
   });
   }
function move_sidebar( class_name ){   
   document.querySelector('.sej-sect>div').classList.add(class_name);// add class 
   //check if clarity has loaded and set tag "experiment" with values "right_sidebar" or "left_sidebar"
   if( typeof window.clarity != 'undefined' ){
      window.clarity("set", "experiment", class_name );
   }
   console.log('sidebar position', class_name );
}
</script>

In this case we are manipulating the DOM to change the layout.

In your specific case, you may need to apply different CSS for layout adjustments. You can use ChatGPT as a handy tool to help you with your custom coding.

Over time, when you have enough sample data for your split tests, you can use the Microsoft Clarity “experiment=main_content_left” or “experiment=main_content_right” tag filter to segment your data.

3. A/B Test Menu Labels

Again, we will use the Math.random() function and manipulate DOM via JavaScript.

We would like to test the “Recents” vs. “What’s new” menu label in our website’s navigation bar.

To do this, let’s get the JS path using the browser’s DevTools, as shown below.

We will use the JS path to access elements in the DOM and change the label.

<script>
   //We want to test the menu label for the Latest section in our website's navigation bar
   window.group_name = "Latest" 
   let randomNumber = Math.floor(Math.random() * 2) + 1; // either 1 or 2. 
   //let randomNumber = Math.floor(Math.random() * 5) + 1; // random number from 1-5. use this if you want to run test on the sample of your traffic e.g. on the 25%.
   if( randomNumber == 2 ){
      group_name = "News" // we will use group_name as a label and a custom tag at the same time;
   }
//If DOMContentLoaded has loaded run the code, otherwise attach an event listener   
if (document.readyState === 'complete') {
     change_label( menu_label )      
   } else {
   // DOMContentLoaded event has not yet fired
   document.addEventListener('DOMContentLoaded', function() {
       change_label( menu_label );
   });
   }
function change_label( menu_label ){   
   document.querySelector("#main-navigation > li:nth-child(1) > a").textContent=menu_label;//set label, in your case it will be different
   //check if clarity has loaded and set tag "experiment" with values "right_sidebar" or "left_sidebar"
   if( typeof window.clarity != 'undefined' ){
      window.clarity("set", "experiment", menu_label );
   }
   console.log('menu label', menu_label );
}
</script>

To add your code, you can either add it to the section Paste into your website or use GTM as previously explained.

Note that when tracking with GA4, you must also use a custom dimension.

To get reports in GA4, you need to use “Explorer Reports” and filter your metric by the “Experiment” custom dimension.

Diploma

A/B testing tools can be expensive to buy and they may not always offer the specific features you need to achieve your goals.

For example, none of the A/B testing tools we tested could provide a user-friendly interface for testing different types of ads without custom coding.

However, the methods outlined here may require some effort to learn custom coding.

However, since ChatGPT is available to you as you write code, the process shouldn’t be too difficult.

More resources:

Featured image: Burdun Iliya/Shutterstock