Web Analytics Key Performance Indicators (KPI’s) are critical for breaking through the dataglut spewing forth from your web analytics tool.   I mean there’s a just a ton of data in web analytics, and the majority of it tends not to be very useful or applicable for improving your business performance.  While it’s wonderful to have a tool that lets you cut, cross, and slice loads of data every which way but loose, its can be a real challenge to frame the data or put it in context in a way that helps your business optimize the web site.   That’s why I like KPI’s - they identify meaningful, business-focused relationships in your analytics data.  By understanding KPI drivers, setting expectations for KPI performance, and analyzing your KPI’s toward defined goals for those KPI’s, you increase understanding of data, alleviate data confusion, and provide focus for the usage of your web analytics tool.

For those of you who don’t have a KPI strategy or who are just getting into analytics, an easy way to understand a KPI is to consider the example of when you are driving somewhere and trying to get there within a certain period of time.  If your goals is drive 60 miles (kilometers, my European friends) in exactly 60 minutes, you know that you need to drive 60 miles per hour (or KPH).  If you go faster, you will arrive early, if you go slower you won’t meet your goal and will arrive past your deadline.   So as you travel along the road, you measure the KPI of your speed. That’s what is important to measure on your trip.  Of course you may measure other KPI’s like the amount of fuel left or the miles you’ve traveled… those certainly may be KPI’s you measure.  But you definitely don’t need to measure you compression ratio or oil pressure even though it’s available data from your car.  In the same way, when you are looking at web analytics data, you don’t want to track everything, only those things that are important to your business performance toward goals. 

Several activities can assist the creation of KPI’s.  Here are a few of them:

• Determine the Business Strategy.  Why is the company funding and developing an online mission?  What is the strategy?  KPI’s can help you figure out if it’s working.  To find the KPI’s that will help, the web analyst should be asking the question how can web analytics be used to formulate, implement and evaluate cross-functional decisions that will enable an organization to achieve objectives? How will web analytics be used in the process of specifying the organization’s objectives, developing policies and plans to achieve these objectives, and allocating resources to implement the policies and plans to achieve the organization’s objectives?
• Define the Site’s Goals and why the Site Exists.  I covered this in a post a few months ago.  A understanding of why your site exists enables you to effectively use online metrics.  You need to define the purpose of your site in order to create effective KPI’s.  Once you’ve defined your site’s purpose, you are positioned to examine Web data in way that helps you determine whether your site delivers on its purpose — does it exist effectively?   Create your KPI’s, identify goals for your KPI’s, and track your performance against those goals.
• Recognize Value Drivers.  How does the business make money on the site? Monetization, in cases where profitability is important, influences what you should be measuring.  If you run a media site, you probably make money from content consumption (the recency and frequency of content consumption), conversation (social media, such as contributions or comments), and conversion (the rate at which people complete certain value driving actions, like signing up for newsletters, rss feed, webcasts, print subscriptions, or downloading certain content types, like white papers).  So you create goals for and measure KPI performance around those value drivers.
• Map Organizational Roles.  Classify your organization into audiences for your KPI’s based what they do on your web site.  You may create KPI’s around function or action of the actors who receive your KPI reports.  Function defines the group that KPI’s are focused for, such as product development or editorial.  Action defines what those people do on the site to make it successful.  By understanding function and action of key actors on your sites, you gain insight into the type of data needed in KPI’s and the number of different KPI reports you may need to roll out.
• Understand the Customer.  KPI’s purely focused on internal function and actions are important, they need to be customer focused.   If you think measuring conversion is important, while your customers tend to come to your site for informational or non-transactional purposes and then go elsewhere to convert, you may be disconnected from the reality of why your site exists.   Learn customer goals from VOC (voice of customer) data and by examining historic behavioral data of key segments.  Make sure you don’t create KPI’s that are vain or inane.  Instead create KPI’s that help you guide action internally so that your business meets the needs of your customers.

Framing your KPI development around the five bullet points I listed above will help you create KPI’s that assist your team in guiding business performance toward goals - while not forgetting to consider some of the core elements of online business: business strategy, site performance goals, value drivers, the human organization, and the customer. 

Now segment, segment, segment your KPI’s!

The infoglut in web analytics is enormous.  So much data.  Companies report that 69% of all people who consume the data don’t understand it.  How does a business go about making sense of it all?  Formulating a comprehensive KPI (Key Performance Indicator) strategy is a big part of differentiating signal from noise and directing appropriate tool usage.  We’ve all heard about KPI’s before.  They are ratios or derivatives of metrics that pinpoint critical, business relevant web performance.   My good friend, Eric, even wrote a book (a BIG one) about it. 

The process of moving an organization through KPI Change Management starts with a well formulated plan for doing so.  Here are some tips for formulating your KPI plan: 

• Educate senior management and get managerial buy-in.  Education and buy-in can take shape via a number of methods.  Maybe you publish and circulate an internal-only white paper about the importance of KPI’s measurement.  Maybe you leave Eric’s book on the chair of your C-level executives.  Perhaps you hold a meeting and present the web site optimization process and how measurement via KPI’s provides the foundational informational on which to make site optimization decisions.  Perhaps you take your boss out to lunch and explain that basic reporting and tool access is helpful, but “Web analytics is hard” and that KPI’s give context to the data to staff that’s otherwise somewhat confused about what they pull for the tool.  You explain that KPI’s provide a focal point for centering analysis around business goals.  Whatever the method, the goal is managerial approval that “yes, you can do KPI’s.”
• Determine the audience for the KPI’s and train them.The importance of KPI’s will vary by stakeholder, and your KPI strategy needs to take that into account. Different segments of stakeholders will be interested in specific KPI’s, and you must accommodate that need.  As an analyst, you should identify the functional roles and job responsibilities of the people who are going to receive KPI reports.   Everyone may not be the right choice (though it could be), and it may make sense to concentrate a KPI rollout on the needs of the few or it may make sense to “go broad.”  Follow up with comprehensive training about your KPI project and how KPI’s can most effectively be used.
• Start with simple, well-qualified, highly relevant KPI’s.  While some folks with want to throw a “kitchen sink” strategy at KPI’s.  That’s a mistake.  If you report more than 5 to 10 KPI’s (imho) per stakeholding group you may end up with a set of unworkable, confusing, and neglected reports.  It’s better to report just a few, well qualified, highly relevant KPI’s.  How do you qualify them? By mapping KPI’s to important business objectives.  How do you know they are highly-relevant? Because you’ve compelled management to buy-in and to agree that they are critical indicators of site success. 
• Elicit the business goals for the KPI’s, compare KPI’s to goals, and report associated variances (i.e. deviations). Make sure you have determined business performance goals for KPI’s.  Goals give context for performance. It’s that simple.  Without goals, you have no context for determining what’s good and what’s bad.  If your conversion rate KPI is 5%.  Great!  So what though?  If you know your goal is 3%.  Awesome job.  If you know your goal is 10%.  Stop reading now, and get back to work - you have much work cut out for you. 
• Identify the frequency and format for reporting.  You need to determine a frequency that is timely and sustainable, and the format in which you present KPI reports needs to common enough that people can easily examine the data. Perhaps you deliver the reporting in Excel, make it available directly in your tool, use Xcellius, or create reports using a BI tool. 
• Automate the delivery of the reporting.Without automation, you may put on the Report Monkey suit and enter Excel hell.  Critical to the successful rollout of any KPI reporting is an automation plan.  Do you email reports, put them in a shared directory, create a set of reports in the tool and provide access, or deliver them in weekly presentations?  The best choice is the option that gets people to use them, listen, and understand what you are trying to do with KPI’s.
• Following the reporting up with analysis and guidance.  Depending on the size and scale or your organization and the resources you have to work with, it may not be possible to provide every stakeholder with detailed analysis.  But you need to do your best to follow up KPI reporting with true analysis and guidance.  Why are KPI’s going up or down?  What are the drivers of the changes? 
• Segment, segment, segment. Site level KPI’s are helpful in understanding overall audience and customer behavior, but they hide important details.  When you slice a KPI by a specific segment, you will realize insights that help you conclude what action to take next.  Overall site repeat visit rate is 37%, but the repeat visit rate for customers who use your “product lookup tool” is 96%.  What does that data indicate about how you market the site, or about why people are coming to the site? 
• Test, test, test.  As you measure > report > analyze > guide based on KPI’s you will undoubtedly determine actions to take on the site.  You should be testing the hypothesis behind these actions via controlled experimentation.   

There’s obviously a lot more to talk about here - from what constitutes a good KPI, to what types of KPI’s different stakeholders should examine, to what are the best KPI’s for particular site types and more.  I guess there’s more blog posts for that, but in the meantime I hope you’ve found this blogviation useful.  Let me know if you have any thoughts to share.

At some point in the career of a web analyst, you will be asked to investigate, assess, and possibly judge the current state of how a company “does” web analytics.  What are some of the areas you should ask about?  Here are some thoughts and a few questions to ask to help inform your analysis (and grease your mental gears):

• Business strategy.  Why does the organization do web analytics?  What’s the goal of having a web analytics team?  Who defines the strategy?  What is the strategy?
• Analytics organization and team structure.  Who is the chief owner of web analytics?  What does the analytics team look like?  How has the team structure been formalized in the organization?  Is the web analytics team effectively staffed and have enough control over resources to do the job?
• Process.  What analytics processes have been defined?  How does a site or site feature progress from not being measured to being effectively measured?
• Data collection. What methods for data collection are being used?  How much data is being collected, and for how long is it stored, and at what level (i.e. detail, aggregate)?
• Reporting.  What data is reported?  What do the reports look like?  Who creates them?  How are they distributed, and in what format?  To whom?  When?  How?
• Analysis.  What’s the difference in this company between reporting and analysis?  How is analysis communicated to stakeholders?  When?  How?
• KPI’s.  What Key Performance Indicators are you measuring?  How are they relevant to the business?  What actions have people taken from KPI analysis that improved business performance?
• Segmentation.  What audience and customer segments exist?  What audience and customer dimensions and attributes are segmented?  Why are they meaningful to the business?  What has the business learned and what action has been taken from the current segmentation analysis strategy?
• Technology.  What analytics technologies are being used?  What does the schema for web analytics look like?  What homegrown technologies are used?  What external technologies have you bought or deployed for analytics?
• Integration.  How is web analytics data integrated with other internal and external data?  Is it integrated with other systems, how? 
• Site Optimization.  Does the company test landing pages, and/or use AB or Multivariate testing software?  If so, whose software, and what business gains have been realized?
• Advertising/Advertisers. How is analytics used to inform or enable advertisers and advertising?
• Privacy.  What safeguards does the company take in protecting analytics data? 
• Qualitative Data.  Is qualitative data contextualized with web analytics data? Do you capture voice-of-customer data?  Use Net Promoter Scores?  Have a research department?  Does web analytics collaborate with research? 

Those are just a few questions to ask.  Many others can be asked.  What would you want to know, and what would you ask?  Please leave a comment.  I’d love to hear your thoughts.

Now for some random thoughts:

• News from Orem.  API / Fusion / Video Tracking… cool.  I’m pretty psyched that Omniture announced a web services API.  That’s fantastic, and confirms how truly important integration is now and will be in the future for analytics data (as I’ve been saying for years… Google will be next). 

Omniture has announced a new methodology, Fusion, and improved capabilities for tracking video.  All sounds very exciting.  But, like Eric, I’m wondering what revolutionary new methodology Fusion really is?  Or is just what Eric’s been saying for the last 4 yearsbranded by Omniture and delivered by the Great Belkin? 

Regarding the video capabilities, I haven’t seen a real demo yet, but I wasn’t immediately impressed with what I saw on my friend Marshall’s blog.  Instead of quartile tracking, it seems like you track the playhead (the part of the video playing) across audience aggregates in increments of one-twelfth, and you get some bubbly visualization (what would that look like with 10,000 videos on your site?), and better access to forums.

I’m hoping I haven’t seen the whole ball of wax, and I look forward to Omniture giving me the grand tour. 

But for a playhead visualization, I was much more impressed with what I saw from Visible Measures and their engagement curve.  And what the heck are those folks at Divinity Metrics up to for measuring video? 

• News from Novato.  One of my favorite gangs of web analytics folks reside in Northern California.  My colleagues at Semphonic have just released a rather impressive “Omniture Implementation Toolkit.” 

I was able to procure a copy, and I’m totally impressed.  It’s full of hard-learned and hard-earned real world practitioner knowledge.  If you are trying to implement Omniture, it is well worth the money. 

Now I’m not sure if this document competes with or acts as a companion to Fusion.  All I can say is that I know the folks at Semphonic are smart, savvy, and very experienced, and there are thousands of Omniture customers out there who could benefit from this document.

• X Change Conference.  I am totally excited for X Change brought to us this year by Semphonic and Web Analytics Demystified.  The last X Change in Napa at COPIA was one of the most intimate, educational, stimulating, and enjoyable conferences that I’ve been too (and did I mention the wine?).  It was pure “class” all the way (in both the sense of style and learning, and did I mention the wine? ;-). 

This year attendance is limited to 100 folks (99 if you count me ;).  Last year, I huddled on “Deploying Measurement Systems in Globally Distributed Enterprises.”  

If you aren’t familiar with X Change or Semphonic  check them out, and make sure to read a few of my favorite bloggers - the prolific deep thinker and expert Gary Angel, the always impressive (and fun) June D(ershewitz), and bright author and web analytics veteran, Phil Kemelor.

Every month I write a column for MediaPost’s Metrics Insider.  This month I wanted tackle my evolving take on Internet video measurement.  Very few companies offer solutions in this space.  Only a few are really differentiated.  Check out Visible Measures, NedStat, TubeMogul, Divinity Metrics, and the usual suspects, Omniture, Unica, WebTrends, ComScore, and Neilsen NetRatings. 

Here’s my column:

IN LATE 2007, THE DIGITAL Video Barometer Executive Survey indicated that more than 80% of media and entertainment executives believe tracking, measuring, and monitoring Internet video content is critical to bottom-line profit.  That’s not surprising. Accurate measurement informs decision-making and improves business performance, and Internet video is more mainstream and popular than ever before.  What may be surprising to those executives is that technology for measuring Internet video generally focuses on video content served on-site, not off-site.  It’s fairly straightforward for a Web analytics tool to tell you how people are consuming and interacting with on-site video, but consumption and interaction of videos distributed across multiple sites, perhaps virally or via social media campaigning, aren’t directly measurable by Web analytics tools.  Panel-based technologies can approximate certain off-site measures of video consumption and distribution, but don’t provide very deep on-site metrics. Measurements of Internet video consumption, interaction, and distribution may be categorized as follows:

• Instream measurement.  Refers to measuring the video itself and the various events and behaviors that occur during a video viewing experience, such as time-based duration metrics and interaction and behavioral metrics (for example, the number of stops, plays, pauses, rewinds, fast-forwards, sites that posted or syndicated the video, clicks on hotspots and social media features).
• Outstream measurement.  Refers to measuring the content environment and user experience surrounding the video on the site or in the skin, such as the conversion metrics (percentage of visitors downloading or viewing a video), source metrics (refers to the video page, players used), and content metrics (percentage videos viewed by topic, percent videos viewed by file type). 

Those categories form a framework for Key Performance Indicators (KPI’s) that help to identify how people interact with videos, how videos perform when compared to other videos, and against pre-defined business goals.  Analysis of KPIs enables video content to be tailored to maximize performance.  Example KPI’s include:

Instream KPI’s:

• Percent high, medium, and low duration video views
• Average viewing time per video
• Percent visitors who complete the video
• Percent visitors that stop the video within 10 seconds
• Percent visits when this video was the last video viewed
• Percent visits when this video was the first video viewed

Outstream KPI’s:

• Conversion rates by video, topic, channel, taxonomy node, referrer, geography, keyword, and so on
• Average video views per visit
• Percent visits/views from different channels (such as email/rss, organic search, paid search, direct)
• Average time between visits that include a video view
• Repeat visit rate for visits involving a video view or download

These KPIs are measurable using a Web analytics tool, and perhaps a few of them are possible using traditional panel-based measurement.  But if off-site video distribution creates a whole new set of challenges to using current analytics and audience measurement tools to track instream and outstream metrics and KPIs, what are publishers and advertisers to do?  It’s a business problem that demands a new technology solution for understanding audience behavior, consumption, and distribution patterns of off-site syndicated or viral video content.

So what would a new technology solution for measuring Internet video and audience behavior do?  First it would have to fill the gap between panel and census-based measurement systems in a way that helps both publishers and advertisers  – not just one or the other — understand audience reach, frequency, and behavior.  The technology must enable tracking and actionable reporting and dashboarding of key metrics and KPIs, distribution patterns, behaviors, and interactions regardless of where the video “goes” on the Internet.  Audience characteristics from external databases (like OpenID for example) and internal company databases (like subscription and registration dbs) should be able to be integrated with data collected about behavior, video metadata, and instream and outstream metrics. 

If measuring digital video is as important as eight out of 10 media and entertainment executives believe it to be, there are some huge money-making opportunities on the horizon — for companies that are already providing technology for tackling this emerging business need, for advertisers using Internet video to drive awareness and response, and for measurement professionals who can help make sense of the Internet video ecosystem, solve measurement challenges, identify significant business opportunities, and use video metrics to improve business performance.  We’re certainly at the beginning of the J-curve for Internet video measurement for both publishers and advertisers.  After all, Forrester predicts Internet video advertising spend to increase from $471 million last year to $7.1 billion in 2012.