There’s been a new round of industry outrage recently for the horribad way that most digital media sites look at metrics.
And that’s great.
Digital media generates a lot of data. The disruptors, pure play digital companies, are generally adept at leveraging this. The disruptees, media companies with their roots in “traditional,” analog media? Not so much. Better use of digital data is a critical step for “old” media companies navigating their transformation to the new world, and it’s a struggle.
Using raw pageviews as the primary metric shows a fundamental misunderstanding of the nature of digital media – it’s an attempt to overlay reach-based metrics that worked for broadcast and print media onto a format where things like relevance and context, not reach, are the drivers.
Worse yet, tunnel vision on pageviews can warp the purpose of a news site, even those without their roots in analog paper and ink. In a thoughtful rant from former VentureBeat writer Bekah Grant earlier this week, “Confessions of an ex-tech journalist,” she says:
The need for speed and volume is primary driven by one thing — pageviews. Pageviews are what sell advertisements, and advertisements are what keep most online publications running — particularly the small independent ones. Are they a good barometer for quality? No, but the reality of online journalism is that you need pageviews to survive.
In a perfect world, important stories would attract the most pageviews, but that is not the world we live in. Miley Cyrus and cat videos get more pageviews than stories about homelessness or healthcare. To write the stories you want, you have to feed the machine. And the machine likes junk food.”
The outrage has a purpose. So far so good.
One problem, though, is that when it comes to metrics, the traditional media industry often gets stuck trying to evolve from single-cell protozoan (“More page views now!”) to advanced space-faring civilization (“We need to know our impact on society!”) in a single step. Without the patience to crawl up out of the sea and learn to walk on dry land first … well, let’s just recognize that despite myriad efforts to evolve, pageviews are still the dominant metric in the industry.
(The trap of trying to go from zero to nirvana in one step is a seductive one, one I’ll readily admit having fallen for myself in the past. I speak as a fellow traveler.)
The current round of kvetching
It’s a great piece of web metric mythbusting. Some of the insights: People frequently share articles they don’t read, attention is more important than clicks, and a lot of content consumption happens “below the fold” — basically an acknowledgment that media companies pack so many ads and static junk at the top of their web sites that users are conditioned to scroll down for content.
Rick Edmonds used Haile’s piece as inspiration for his rallying cry in Poynter on Monday of this week (3/17), “Time to ditch uniques and page views for engagement in measuring digital audience.” Attention and engagement are the two measures Edmonds focuses on the most.
Lindsay Green-Barber’s post for Nieman Lab followed on Tuesday, “How can journalists measure the impact of their work?” She makes the case for engagement and impact as the key metrics. She acknowledges that there is no agreed on way to measure such things yet, but points to some complex aggregate metrics in development at places like The New York Times, citing work by Brian Abelson.
Abelson is smart. He’s a stat geek’s geek, and if you’re into that (I am), his posts at www.brianabelson.com are a great read. Green-Barber cites Abelson’s metric for engagement, from his 2013 post “Creating a metric for news apps,” as an example of where things could go. (I’ve included it here.)
So where’s the problem?
Attention? Engagement? Impact?
Those are sound like great things to measure. So why don’t we just measure them, ditch pageviews, and be done with it?
For starters, because metrics for them don’t exist.
We can try to approximate attention by measuring time on site. But as many have pointed out (as recently as 3/10/14 by Jeff Jarvis, building off Haile’s post in “Good metrics, bad metrics“), time on site isn’t necessarily a measure of a great experience. If you have to hunt around a site for a long time to find the information you’re looking for, time on site goes up, but it’s not a positive. If you come to an article page via a Facebook link, spend 15 minutes enjoying that article, then close your browser tab, Google Analytics will record your visit as zero seconds in length, since Google can’t measure time spent on the last page you visit, just time from one click to the next.
Engagement is famously hard to measure. Aggregate measures like Abelson’s are fascinating to the stat geek in me. But aggregate measures are problematic because (1) aggregate metrics are notoriously hard to get people to believe in, and (2) aggregate metrics are notoriously poor for driving action. Did your custom multi-variable engagement metric drop? Why? Which one of the five, or seven, or 12 variables moved? Did more than one of them move?
The primary purpose of analytics is to drive action, and metrics that require additional analysis before actionability can be understood aren’t generally great solutions. Even more so for real time products like news sites.
And impact? We all want to know that. But it’s even harder to measure than engagement.
So, no rocketships. How do we start walking?
Yes, anybody can be a critic. Where can you start?
Maybe here’s a few principles to build from:
1. Analytics don’t matter unless they drive action. To do that they need to be simple so they are understood and tied to things people can do. Focus on simple and actionable first.
2. Start with the actions you want people to take. What are, to borrow from Clay Christensen, the “jobs to be done” by the news site?
3. Understand that the villain in this story, the dreaded pageview, isn’t always bad. Just misunderstood. (Pageviews don’t kill people, people using pageview metrics irresponsibly kill people.)
Avinaush Kaushik’s Web Analytics 2.0 came out in 2009, but is still cited as the go to first stop by many digital gurus if you want to establish a good, foundational understanding of web metrics. (Kaushik is Google’s Digital Marketing Evangelist and co-founder of Market Motive, and he blogs at Occam’s Razor.)
It’s a textbook length tome, and more than can be recapped here. But keeping things simple and actionable – making the data seem simpler, not more complex – is a huge emphasis.
In Kaushik’s view, pageviews aren’t bad, as long as you’re segmenting them to get at the information you want.
Total pageviews aren’t meaningful. But the number of visits where readers stayed for nine or more articles (“depth of visit”) might be. Find ways to lead readers from one article to another (USA Today’s site redesign last year focused on this) if this is the behavior you want.
Total visitors may not be that meaningful, but number of visitors that come to the site at least 4 times a week may be a good indicator of how relevant you are to your audience. Campaigns to sign readers up for daily emails that recap key headlines can help encourage this behavior.
At last year’s NAA MediaXchange, Chris Hendricks of McClatchy made the excellent point that local news companies should always segment out local audience, because local audience is what advertisers are paying for and who the newsroom serves. That’s also a simple task in any analytics platform.
Just as importantly, don’t rely solely on clickstream numbers in your analytics packages to give you all the insights. Clickstream metrics (the kind of metrics you find in Google Analytics) are great at telling you the “what” but not the “why.” Kaushik is a big proponent of adding “Voice of Consumer” tactics — surveys and other forms of good old analog era market research — to add context.
Kaushik’s favorite “Voice of Consumer” tactic — the one he would do if he could do only one — is to add a 3 question survey to the end of a site visit.
- What is the purpose of your visit to our website today?
- Were you able to complete your task?
- If you were not able to complete your task today, why not?
Simple, direct, and rich with context that will help you understand the meaning behind those clickstream metrics. Basically: Are you accomplishing your “Tasks to be Done”?
Perfectionist cultures are common in news companies. It’s not surprising, given the huge importance placed on accuracy in a newsroom.
But it’s a trap to wait for the “perfect” metric before switching away from total pageviews. The perfect metric doesn’t exist. Rocketships will be built eventually. For now, start walking.