Analytics, Ethics and Leadership: Dialogue with the Dean

August 30, 2016

The New York Times calls its custom-crafted dashboard “Stela” — which stands for “story and event analytics.” According to Shan Wang’s report for Niemanlab.org, the Times makes this user-friendly system available to staff so they can see an array of data about their articles:

“We were looking for ways to help reporters and editors get feedback on the things they were being asked to do online, such as tweaking headlines, promoting to social,” Steve Mayne, lead growth editor at the Times, said. “And we believed it would be much more effective for us to actually have a tool to show reporters how, for instance, certain actions directly resulted in more people reading their stories.”

The system as described by Wang is impressive and effective, and has become fairly well adopted inside the Times. As media organizations gain greater access to these instant report cards, several questions arise:

• Is widespread attention to analytics compatible with good journalism?  
• How should media leaders make certain the data isn’t misused?

Loyola’s Don Heider (School of Communication Dean) and Jill Geisler (Bill Plante Chair in Leadership and Media Integrity) sort it out.

Don Heider: I think in this case, like so many, context is key.  I can see using analytics as is described in the NY Times piece to really help reporters and editors be more responsive to the audience.  I think most of us at this point realize that journalism today and in the future must be more interactive, and this gives journalists a tool set to pay attention to how readers are responding to their stories, headlines, and even photos and videos.   The worry is of course is about the “P” word.  Will journalists begin pandering to readers to try to build views and clicks?  When I said context above, I meant context as is in; who is in the newsroom?  If you have a veteran crew of writers, reporters and editors, I think there is little risk.  Managers can help by making sure the mission of the organization is clear, and even what goals are when using analytic information. What are you seeing among the managers you teach and coach in news organizations?

Jill Geisler: Managers vary greatly when it comes to analytics. Some are protective of performance data - just because they like to control the flow of information in general. Some are conservative about sharing,  fearing it will be misinterpreted and cause other “P” words like “panic” or “paranoia.” Some are still learning analytics themselves. And then there are folks like my friend Marty Kady, editor of POLITICO Pro. Here’s what he told me:

“On my team, I've gone fully in favor of providing metrics (though we don't judge our paywall products by total clicks). We have provided open rates for email newsletters and alerts, subscription renewal rates and a full list of subscribers to all the section editors. If you want people to feel fully bought in to the news and product mission, I think transparency in how we're doing is essential."

I like Marty’s transparent approach. With transparency comes additional responsibility for leaders.  To share analytics effectively, think: Strategy, Success and Soul. Explain your organization’s strategy and how the metrics support it. Define clearly how the metrics do or don’t measure the success of the whole team and each individual member. Never forget that data-driven organizations can easily lose sight of values, their soul - without strong leadership. Here’s my at-a-glance guide for sharing analytics:

That said, let me ask you, Don, for your take the biggest ethical land mines you’d encourage media organizations to guard against when it comes to analytics? What’s your top five list?

Don Heider: I don’t know about a top five, but here are things I think about: It sounds like Politico has an excellent approach.  But do most newsrooms have the resources they to help put metrics into context? As I was saying above, I worry that analytics without context can lead journalists to conflate popularity (impressions, page views, etc.) with journalistic importance.  We always have to come back to that question; what’s our journalistic purpose?  Why we journalists and what are is our duty?  I would argue, even in a digital click-through age, our duty is to inform people, serve as watchdogs, and to tell important stories well.  There are times when the most important stories do not perform as well as the less important stories (such as the latest Kardashian saga).  That never releases us from our obligation to try to do our best to inform. We can use analytics to helps us gain a broader understanding of what the public wants and needs to know, but we have to dig a little, examine trends and even ask the public from time-to-time; page impressions does not do that effectively.  The bottom line; analytics have to be aligned with journalistic purpose.  Conversely, following the wrong metrics can lead journalists in the wrong direction (Buzzfeed’s clickbait comes to mind). As a researcher I can also tell you that one set of data never tell you the whole story.  There are always hundreds of variables that can influence an outcome and this definitely holds true with web analytics.  Most often a data set tells you what, it almost never tells you why. Web analytics will never replace a human being’s ability to develop sources, ferret out a story or witness an event.

Computers, algorithms, data analysis all become really helpful and powerful tools when paired with human intelligence. I also think the more we look at analytics the more we realize that the future of journalism will be based upon building relationships with our audience.  Engaging people in what we do, including listening to their ideas and feedback, even meeting them face-to-face.  I think if we can really engage people in what we do and how we do it, there’s more chance they will financially support our endeavors. Finally, I worry that if newsrooms become overly dependent on metrics, it may discourage risk-taking.  We don’t want to get into the well-worn grooves of doing what works over and over.  I have often see a crazy idea do more to break new ground and engage people than just repeating the same kind of news over and over again.