GA4-Driven Experience Insights

This document details an older version of Experience Insights that relies on sending Ninetailed Experience impressions and conversion events to Google Analytics 4, then forwarding them onto Ninetailed using BigQuery exports. Both new and existing Ninetailed customers should set up "native" Experience Insights by 1) configuring the Insights Plugin and 2) sending conversion events with track method exposed in Ninetailed's SDKs. See the Experience Insights setup guide for more information.

Ninetailed's Experience Insights integrates with your existing analytics tool to provide statistical analysis of the success of your experiments and personalizations against metrics that you already measure.

Step 1: Setting Up the Data Flow

Currently, we support the setup of Experience Analytics only in conjunction with Google Analytics 4. We are currently developing a way to integrate with other tools as well. If you have any requests to support a specific analytics tool, please contact us.

Experience Insights leverages the complete dataset of the events captured by your analytics platform to attribute experience variants to conversions or goals. Ninetailed’s nt_experience events indicating that a particular experience and variant have been seen must also be sent to your analytics platform. Afterwards, that data can be forwarded together with your conversion events.

Sending Ninetailed events to your Analytics platform

In order to attribute a conversion goal to the an experience impression, we need to forward this information to your Analytics platform. We recommend using the Ninetailed Google Tag Manager plugin for this. Refer to the developers installation guide to provide the plugin in your application. Next, you'll configure variables, tags, and triggers within your GTM container.

1. First, set up your Ninetailed variables in Google Tag Manager. The required variables are:

• ninetailed_experience

• ninetailed_variant

• ninetailed_audience

• ninetailed_component

An example can be seen here:

Variable configuration in Google Manager

2. Set up your Trigger event, that fires when an Experience was seen. To do so, add a Trigger with the event name nt_experience

Trigger configuration in Google Tag Manager

3. Lastly, create a Tag that sends all your variables to your Analytics platform. It is important to exactly match the event name and parameter names. The following parameters should be sent to an event called nt_experience:

   1. ninetailed_experience

   2. ninetailed_variant

   3. ninetailed_audience

   4. ninetailed_component

The tag type depends on what analytics platform you want to send that tag to. In most cases, it will be a “Google Analytics: GA4 Event” type.

Tag configuration in Google Tag Manager

Connecting GA4 Data to Ninetailed Statistics Engine

In a next step, we are need to grant Ninetailed permission to read raw events from your analytics tool. For that we are leveraging BigQuery exports, copying the data from your BigQuery dataset into our own instance on a daily basis in order for us to perform statistical analysis.

Steps

1. Your GA4 data needs to be exported to a BigQuery Dataset which can be consumed by Ninetailed and copied over to the Ninetailed Data Cloud. If you do not already have a BigQuery dataset connected to GA4, complete the steps outlined in the GA4 documentation how to set up GA4 exports to BigQuery.

2. Next, go to your Google project, click on the navigation menu on the top left, then hover over IAM & Admin. In the menu that slides open, click on the IAM option at the top of the menu.

   

   IAM options in GCP
3. In the IAM section, you should be able to see a “Grant Access” button. Click on it.

   

   Granting access in GCP
4. This time a page slides open from the right of the screen. This page is where you will add the service account. Service accounts are identified by email addresses and have “roles” attached to them. This screen allows you to enter both these values.

   1. In the “Add principals” section, type [e-mail] and press enter.

      

      Adding a principal in GCP
   2. In the “Assign roles” section, click on the drop down. It should open a popup as shown.

   

   Assigning the correct role in GCP

   Google will suggest some popular roles will be suggested by default. The role we are looking for is BigQuery Data Viewer. If it does not show up, you can type it in the filter text box located right at the top of the popup.

5. When you are done adding both the principal (service account’s email address) and the role, your screen should look like this:

   

   Final access settings to connect BigQuery with Ninetailed in GCP
6. Click on the “Save” button to finish.

7. The service account should be added to your project.

   

   Final service account in GCP
8. Once the service account is added, please reach out to your point of contact at Ninetailed with two key pieces of information:

   1. The project id which owns the BigQuery instance.

   2. The BigQuery dataset id which holds the exported GA4 data (Example: ninetailed-project.analytics_456712034)

If you have any issues or questions in the steps above, feel free to contact us any time.

Step 2: Setting Up Performance Metrics

This document details an older version of Experience Insights. For up-to-date information, please visit Setting Up Metrics.

Metrics are conversion events or goals. By default, Ninetailed tracks the performance of all metrics that you have set up across all of your Experiences.

Example: you can track the number of newsletter signups or submissions to a form, the order quantity or purchase revenue that a user generates or even just a scroll to a specific position on a page.

Experience Analytics is leveraging the fact that you are already tracking conversion events, goals, or other metrics in a different analytics platform. Therefore, forwarding these metrics to also have them available for measuring the performance of Experiences is very easy. All you have to do, is tell us what your event is called, and what it measures. Specifically, there are two elements that need to be configured to measure your events correctly: the metric type and the metric scope.

Before you set up any metrics at Ninetailed, make sure that you are already tracking a corresponding event in your connected analytics solution.

Metrics in Experience Analytics

Metric Types

This document details an older version of Experience Insights. For up-to-date information, please visit Setting Up Metrics.

There are two distinct metric types. Choose the right one depending on what you want to track.

Conversion Rate

This metric type tracks the success rate of users “converting” (i.e., triggering a specific event) after having seen a specific Experience. The result is a percentage value ranging from 0% to 100%. Use this metric type for any binary goals.

Examples for metrics where the "conversion rate" type should be used:

• “a visitor has signed up to newsletter”

• “a visitor has submitted this form”

• “a visitor has made a purchase”

Conversion Value

In some cases, the value of a conversion or goal is also relevant. This metric type tracks any given value of a conversion event. We can differentiate between a number of different value types. Depending on what you choose, we will reflect them differently in our analytics interface.

Examples for metrics where the "conversion value" type should be used:

• Currency values - e.g., “Purchase value”

• Time values - e.g., “Session time” or “time spent in checkout”

• Normal values - Any other non-formatted value, e.g., “items in the basket”

Metric Scope - Success Attribution

This document details an older version of Experience Insights. For up-to-date information, please visit Setting Up Metrics.

In order to track the performance of Experiences, we need to identify at which point conversion or goal should be attributed to having seen an Experience. At Ninetailed, we differentiate between two distinct types of attributions: Session Scope and User Scope. Which scope should be used depends on the use case and the type of Experience that you are running.

Success attribution (or scoping) is set on a metric-level. This means that different metrics can measure the success of specific Experiences at different scopes simultaneously.

The scope of the metrics decides whether the total amount of sessions or the total amount of users are counted as "Reach".

Session Scope

When choosing “Session Scope”, any Experience that has been seen in the same session and before reaching a conversion goal is attributed towards reaching that goal. This type of attribution goal is useful when a conversion usually happens within the same session or if your Experience is clearly directed towards a specific event such as clicking a CTA.

Example: An Experience that shows a CTA button to check out a specific site mostly contributes to the session behavior of that user. If we are tracking a "button click" as a metric, we should use session scope for that.

Example: An Experience that suggests an item that the user might want to buy with the items they already have in their basket contributes towards increasing purchase revenue within the same session. Measuring "purchase revenue" on a session scope makes most sense here.

User Scope

When choosing “User Scope”, any Experience that has been seen at any point in time before reaching a conversion goal is attributed towards reaching that goal.

This type of attribution is most useful in situations where users tend to reach a conversion goal only after several sessions.

Example: Experiences that drive users to sign up to a form contribute towards reaching that goal even across sessions. If we are tracking that final "form submission", we should use a user scope for that.

Example: Any Experience that improves the user experience, in general, generates user retention, which has an effect across all user sessions. If we are tracking a final "demo request" as a metric, we should be using a user scope for such Experiences.

The setup of Experience Analytics follows the same principles and core values of Ninetailed, integrating within the tools you already use and not creating another source of truth for metrics that you already measure.