A Beginner’s Guide to Cohort Analysis: the Most Actionable (and Underrated) Report on Google Analytics
A Beginner’s Guide to Cohort Analysis: the Most Actionable (and Underrated) Report on Google Analytics

The antidote to vanity metrics when you have less than 1 hour/week for marketing analytics

The Class of 2017 is an example of a cohort. All these students share the same date of graduation.

Earlier this year, we at Humanlytics interviewed more than 100 small and medium-sized businesses (SMBs) to learn about their biggest digital marketing pain points. What we found was that two recurring themes kept coming up over and over again:

Scarce Time/Bandwidth for Analytics: Most small business owners and marketers have very limited time and bandwidth to analyze their data. They’re too busy running their businesses. As one business owner put it, “using data is like trying to rescue a piano from a burning room — it’s nice to have but I have higher priorities.” This is because it takes a lot of time and training to turn data into actionable business insights, i.e. to answer business questions and support business decision-making. Vanity Metrics Don’t Add Business Value: Vanity metrics are metrics that make you feel good about your business, but don’t actually help you make decisions (they’re not actionable). The trouble with vanity metrics is that they can actually be worse than not using analytics. They’re not only a waste of time, they can actually mislead your business decisions. Aggregate metrics like “new sessions” on your website, for instance, can obscure more than they reveal. For example, if your number of new sessions goes up, is this a good thing (getting traction with new users), or a bad thing (retention with returning users is going down)?

Vanity metrics are like instagram filters. They make you look good, but you don’t have an accurate depiction of reality. Image via Giphy.

This is where cohort analysis comes in. The cohort analysis feature in Google Analytics is the antidote to both problems (limited time and misleading vanity metrics).

What is a cohort? In a nutshell, a cohort is simply a subset of users grouped by shared characteristics. In the context of business analytics, a cohort usually refers to a subset of users specifically segmented by acquisition date (i.e. the first time a user visits your website).

A “cohort analysis,” then, simply allows you to compare the behavior and metrics of different cohorts over time. You can then find the highest-performing (or lowest-performing) cohorts, and what factors are driving this performance.

The cohort analysis report is one of the most underrated features on Google Analytics. Why? Because it helps you isolate the impact of your different marketing activities on a specific group of recipients, instead of noise in the data.

Running a cohort analysis is one of the simplest ways to run an experiment for your business. As a marketer, you can run a time-bound campaign with certain characteristics you want to test: ad content, marketing channel, target audience, landing page design, etc. You can then compare metrics for reach, engagement, and conversion for these different marketing campaign, to see which factors of the campaign actually added value to your business, and which didn’t.

Let’s run a marketing experiment! Image via Giphy.

This is the real value and purpose of marketing analytics in the first place. Simply put, marketing analytics tells you what is working, what is not working, and how to adjust your marketing activities based on this feedback. Cohort analysis does just that by focusing on the effect of each marketing activity or change on a specific audience in time.

As for the time scarcity issue for many businesses, going through your typical cohort analysis every week will take you less than an hour. Every business has different needs for marketing analytics. But for many businesses, if you only have the time and bandwidth to look at one Google Analytics report per week, I often recommend starting with the cohort analysis report, instead of wasting time with vanity metrics.

In this article, we’ll cover:

What Cohort Analysis is Good For Limitations of Cohort Analysis in Google Analytics A Beginner’s Primer to the Cohort Analysis Report in Google Analytics A Few Examples of Cohort Analyses to Get You Started

What Cohort Analysis is Good For

As a business analytics technique, a Cohort Analysis allows you to compare variables and changes between your digital marketing campaigns.

For example, like real brick-and-mortar stores, websites change. If you’re doing it right, they change a lot and often. You can use a cohort analysis to try to isolate the effect of the website modification on user behavior.

Here are some factors that can impact user behavior that you may want to analyze with a Cohort Analysis:

Target audience

Ad content

Channels

Campaigns/experiments

Website redesigns

New product lines and service offerings

Sales, discounts, promotion campaigns

Cohort analysis lets you isolate a variable’s effect. Image via Giphy.

In web analytics, you can compare how cohorts perform in traffic metrics (e.g. returning users), engagement metrics (e.g. average session duration), or conversion metrics (e.g. sessions with transactions).

Although you can theoretically analyze any of these factors with a cohort analysis, not every analytics tool (e.g. Google Analytics) allows you to analyze the impact of all of these factors on user behavior.

Limitations of Cohort Analysis in Google Analytics

Although cohort analysis can be very useful in theory, the cohort analysis report in Google Analytics has many limitations in practice.

First of all, cohorts in general can technically be grouped by any shared characteristic. However, the cohort analysis report in Google Analytics (which has been in beta for a while) can currently only define cohorts based on acquisition date (i.e. the first time a user visits your website).

Secondly, tracking retention and returning users on your website (which is what cohort analysis is often used for) is currently an imprecise exercise for Google Analytics. For example, let’s say Pete is a user on your site and visits your site today. If he visits again tomorrow, Google Analytics should register him as a returning user.

However, if Pete does any of these things, Google Analytics may not be able to properly track his next session as a returning session:

Clearing browser cookies

Visiting site on a different device or browser

Visiting the site on incognito mode

The typical digital consumer now owns an average of 3.64 devices, and 36% of Americans own a smartphone, a computer, and a tablet. This inability to consistently track users across devices, browsers, and sessions is not a trivial problem.

Lastly, there’s the problem of confounding variables. As we discussed earlier, it can be helpful to overlay your cohort metrics with your marketing calendar to see how metrics change with your marketing activities.

But any association between a marketing campaign and an uptick in metrics is a correlation, not a causation. Perhaps you started that new Facebook ad campaign last Monday, but can the increase in user retention really be attributed to the quality of your Facebook ads? Or could it be that one of your older blog posts is starting to get traction?

Unless you run randomized controlled trials (RCTs) where you randomly assign users to a control group or a treatment group, you can’t definitively establish causality between a marketing campaign and the metric changes. This is especially true if you have multiple campaigns running at the same time.

That having been said, even with its limitations, the cohort analysis report in Google Analytics can still be directionally correct for data-driven decision-making, especially if you test your marketing campaigns and changes like separate experiments (e.g. if you only run Facebook ads in January, Twitter ads in February, Adwords campaigns in March, etc).

Here’s a walk-through of how to use the cohort analysis feature in Google Analytics.

A Beginner’s Primer to the Cohort Analysis Report in Google Analytics

You can find the Cohort Analysis report under Audience.

At the top of the cohort analysis report, you can adjust settings for cohort type, cohort size, metric, and date range.

Cohort type: currently, the only option is acquisition date (the date of user’s first session)

currently, the only option is acquisition date (the date of user’s first session) Cohort size: you can choose to define cohorts by day, by week, or by month. For instance, if you select by month, then each cohort represents the users acquired in a particular month (e.g. the January cohort includes all the users who had their first session in January)

you can choose to define cohorts by day, by week, or by month. For instance, if you select by month, then each cohort represents the users acquired in a particular month (e.g. the January cohort includes all the users who had their first session in January) Date range : the window of time that you want to examine (e.g. the last 6 weeks)

: the window of time that you want to examine (e.g. the last 6 weeks) Metric: the data you’ll see in report. The default metric is user retention, which measures the percent of users that return.

You can also choose these “per user” metrics and “total” metrics:

You can then select which cohorts to display on the graph.

You can also add additional segments (e.g. mobile/tablet traffic, etc) for comparison, just like with any other report, by clicking by clicking the plus button next to “All Users” at the top of the report. You can find our tutorial on using Google Analytics segments to analyze your audience here:

However, the real meat of the cohort analysis report is the heat map right below this graph. For example, below I compare All Users with the Paid Traffic segment.

The meat of the cohort analysis report in Google Analytics

This heat map allows you to quickly identify the highest (and lowest) performing metrics by cohort and week after acquisition date. Week 0 represents the week in which that cohort’s users had their first session. This cohort heat map cannot be exported, so you may have to copy/paste or screenshot the graphic.

Let’s say that I ran a new Adwords remarketing campaign the week of 9/11 to retarget users who visited my site. As you can see in the Cohort Analysis report above, my user retention went up significantly that week. This may be evidence that my remarketing campaign is increasing user retention, which I can further explore in my Adwords report (under Acquisition).

This is why I recommend pulling up your marketing calendar in a separate window to overlay your cohort heat map with the context of your marketing activities. If you just want to track the dates of your marketing campaigns, I suggest trying the built-in Annotations feature in Google Analytics reports.
['Patrick Han']
['Google Analytics', 'Analytics', 'Startup', 'Digital Marketing', 'Tutorial']